9 papers
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
Shiwei Liu, Tianlong Chen, Zahra Atashgahi +6
The success of deep ensembles on improving predictive performance, uncertainty estimation, and out-of-distribution robustness has been extensively studied in the machine learning l…
Efficient Image Generation with Variadic Attention Heads
Steven Walton, Ali Hassani, Xingqian Xu +2
While the integration of transformers in vision models have yielded significant improvements on vision tasks they still require significant amounts of computation for both training…
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
Chaojian Li, Wuyang Chen, Yuchen Gu +4
Semantic segmentation for scene understanding is nowadays widely demanded, raising significant challenges for the algorithm efficiency, especially its applications on resource-limi…
ShiftAddNet: A Hardware-Inspired Deep Network
Haoran You, Xiaohan Chen, Yongan Zhang +5
Multiplication (e.g., convolution) is arguably a cornerstone of modern deep neural networks (DNNs). However, intensive multiplications cause expensive resource costs that challenge…
Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networks
Haoran You, Chaojian Li, Pengfei Xu +6
(Frankle & Carbin, 2019) shows that there exist winning tickets (small but critical subnetworks) for dense, randomly initialized networks, that can be trained alone to achieve comp…
InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks
Yonggan Fu, Zhongzhi Yu, Yongan Zhang +6
The promise of Deep Neural Network (DNN) powered Internet of Thing (IoT) devices has motivated a tremendous demand for automated solutions to enable fast development and deployment…