144 citations · 237 across the 8 of their papers we have counts for
8 papers
Less Is More: Fast and Accurate Reasoning with Cross-Head Unified Sparse Attention
Lijie Yang, Zhihao Zhang, Arti Jain +5
Large reasoning models achieve strong performance through test-time scaling, but this incurs substantial computational overhead due to long decoding from short prompts. While spars…
Collage: Seamless Integration of Deep Learning Backends with Automatic Placement
Byungsoo Jeon, Sunghyun Park, Peiyuan Liao +3
The strong demand for efficient and performant deployment of Deep Learning (DL) applications prompts the rapid development of a rich DL ecosystem. To keep up with this fast advance…
GradSign: Model Performance Inference with Theoretical Insights
Zhihao Zhang, Zhihao Jia
A key challenge in neural architecture search (NAS) is quickly inferring the predictive performance of a broad spectrum of networks to discover statistically accurate and computati…
Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads
John Thorpe, Yifan Qiao, Jonathan Eyolfson +8
A graph neural network (GNN) enables deep learning on structured graph data. There are two major GNN training obstacles: 1) it relies on high-end servers with many GPUs which are e…
IOS: Inter-Operator Scheduler for CNN Acceleration
Yaoyao Ding, Ligeng Zhu, Zhihao Jia +2
To accelerate CNN inference, existing deep learning frameworks focus on optimizing intra-operator parallelization. However, a single operator can no longer fully utilize the availa…
Redundancy-Free Computation Graphs for Graph Neural Networks
Zhihao Jia, Sina Lin, Rex Ying +3
Graph Neural Networks (GNNs) are based on repeated aggregations of information across nodes' neighbors in a graph. However, because common neighbors are shared between different no…
Beyond Data and Model Parallelism for Deep Neural Networks
Zhihao Jia, Matei Zaharia, Alex Aiken
The computational requirements for training deep neural networks (DNNs) have grown to the point that it is now standard practice to parallelize training. Existing deep learning sys…
Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networks
Zhihao Jia, Sina Lin, Charles R. Qi +1
The past few years have witnessed growth in the computational requirements for training deep convolutional neural networks. Current approaches parallelize training onto multiple de…