305 citations · 659 across the 23 of their papers we have counts for
24 papers
DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase +16
ChatGPT-like models have revolutionized various applications in artificial intelligence, from summarization and coding to translation, matching or even surpassing human performance…
Chiplet Cloud: Building AI Supercomputers for Serving Large Generative Language Models
Huwan Peng, Scott Davidson, Richard Shi +2
Large language models (LLMs) such as OpenAI's ChatGPT and Google's Gemini have demonstrated unprecedented capabilities of autoregressive AI models across multiple tasks triggering…
HEAT: A Highly Efficient and Affordable Training System for Collaborative Filtering Based Recommendation on CPUs
Chengming Zhang, Shaden Smith, Baixi Sun +6
Collaborative filtering (CF) has been proven to be one of the most effective techniques for recommendation. Among all CF approaches, SimpleX is the state-of-the-art method that ado…
Mitigating Coupling Map Constrained Correlated Measurement Errors on Quantum Devices
Alan Robertson, Shuaiwen Leon Song
We introduce a technique for the suppression of state-dependent and correlated measurement errors, which are commonly observed on modern superconducting quantum devices. Our method…
MSREP: A Fast yet Light Sparse Matrix Framework for Multi-GPU Systems
Jieyang Chen, Chenhao Xie, Jesun S Firoz +5
Sparse linear algebra kernels play a critical role in numerous applications, covering from exascale scientific simulation to large-scale data analytics. Offloading linear algebra k…
Towards Efficient Architecture and Algorithms for Sensor Fusion
Zhendong Wang, Xiaoming Zeng, Shuaiwen Leon Song +1
The safety of an automated vehicle hinges crucially upon the accuracy of perception and decision-making latency. Under these stringent requirements, future automated cars are usual…
Brief Industry Paper: The Necessity of Adaptive Data Fusion in Infrastructure-Augmented Autonomous Driving System
Shaoshan Liu, Jianda Wang, Zhendong Wang +7
This paper is the first to provide a thorough system design overview along with the fusion methods selection criteria of a real-world cooperative autonomous driving system, named I…
COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression
Sian Jin, Chengming Zhang, Xintong Jiang +5
Training wide and deep neural networks (DNNs) require large amounts of storage resources such as memory because the intermediate activation data must be saved in the memory during…
Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving
Qiyu Wan, Haojun Xia, Xingyao Zhang +3
Bayesian Neural Networks (BNNs) that possess a property of uncertainty estimation have been increasingly adopted in a wide range of safety-critical AI applications which demand rel…
MAPA: Multi-Accelerator Pattern Allocation Policy for Multi-Tenant GPU Servers
Kiran Ranganath, Joshua D. Suetterlein, Joseph B. Manzano +2
Multi-accelerator servers are increasingly being deployed in shared multi-tenant environments (such as in cloud data centers) in order to meet the demands of large-scale compute-in…
Dr. Top-k: Delegate-Centric Top-k on GPUs
Anil Gaihre, Da Zheng, Scott Weitze +5
Recent top- computation efforts explore the possibility of revising various sorting algorithms to answer top- queries on GPUs. These endeavors, unfortunately, perform signifi…
Randomness In Neural Network Training: Characterizing The Impact of Tooling
Donglin Zhuang, Xingyao Zhang, Shuaiwen Leon Song +1
The quest for determinism in machine learning has disproportionately focused on characterizing the impact of noise introduced by algorithmic design choices. In this work, we addres…
Toward Efficient Interactions between Python and Native Libraries
Jialiang Tan, Yu Chen, Zhenming Liu +4
Python has become a popular programming language because of its excellent programmability. Many modern software packages utilize Python for high-level algorithm design and depend o…
Q-VR: System-Level Design for Future Mobile Collaborative Virtual Reality
Chenhao Xie, Xie Li, Yang Hu +3
High Quality Mobile Virtual Reality (VR) is what the incoming graphics technology era demands: users around the world, regardless of their hardware and network conditions, can all…
Fast and Scalable Sparse Triangular Solver for Multi-GPU Based HPC Architectures
Chenhao Xie, Jieyang Chen, Jesun S Firoz +5
Designing efficient and scalable sparse linear algebra kernels on modern multi-GPU based HPC systems is a daunting task due to significant irregular memory references and workload…
A Novel Memory-Efficient Deep Learning Training Framework via Error-Bounded Lossy Compression
Sian Jin, Guanpeng Li, Shuaiwen Leon Song +1
Deep neural networks (DNNs) are becoming increasingly deeper, wider, and non-linear due to the growing demands on prediction accuracy and analysis quality. When training a DNN mode…
ClickTrain: Efficient and Accurate End-to-End Deep Learning Training via Fine-Grained Architecture-Preserving Pruning
Chengming Zhang, Geng Yuan, Wei Niu +8
Convolutional neural networks (CNNs) are becoming increasingly deeper, wider, and non-linear because of the growing demand on prediction accuracy and analysis quality. The wide and…
MalFox: Camouflaged Adversarial Malware Example Generation Based on Conv-GANs Against Black-Box Detectors
Fangtian Zhong, Xiuzhen Cheng, Dongxiao Yu +3
Deep learning is a thriving field currently stuffed with many practical applications and active research topics. It allows computers to learn from experience and to understand the…
TSM2X: High-Performance Tall-and-Skinny Matrix-Matrix Multiplication on GPUs
Cody Rivera, Jieyang Chen, Nan Xiong +2
Linear algebra operations have been widely used in big data analytics and scientific computations. Many works have been done on optimizing linear algebra operations on GPUs with re…
OO-VR: NUMA Friendly Object-Oriented VR Rendering Framework For Future NUMA-Based Multi-GPU Systems
Chenhao Xie, Xin Fu, Mingsong Chen +1
With the strong computation capability, NUMA-based multi-GPU system is a promising candidate to provide sustainable and scalable performance for Virtual Reality. However, the entir…
Enabling Highly Efficient Capsule Networks Processing Through A PIM-Based Architecture Design
Xingyao Zhang, Shuaiwen Leon Song, Chenhao Xie +3
In recent years, the CNNs have achieved great successes in the image processing tasks, e.g., image recognition and object detection. Unfortunately, traditional CNN's classification…
Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect
Ang Li, Shuaiwen Leon Song, Jieyang Chen +4
High performance multi-GPU computing becomes an inevitable trend due to the ever-increasing demand on computation capability in emerging domains such as deep learning, big data and…
SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks
Linnan Wang, Jinmian Ye, Yiyang Zhao +5
Going deeper and wider in neural architectures improves the accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) pr…
A Graph-based Model for GPU Caching Problems
Lingda Li, Ari B. Hayes, Stephen A. Hackler +3
Modeling data sharing in GPU programs is a challenging task because of the massive parallelism and complex data sharing patterns provided by GPU architectures. Better GPU caching e…