169 citations · 480 across the 9 of their papers we have counts for
9 papers
AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models
Xiaofan Zhang, Zongwei Zhou, Deming Chen +1
Recently, large pre-trained models have significantly improved the performance of various Natural LanguageProcessing (NLP) tasks but they are expensive to serve due to long serving…
MT-TransUNet: Mediating Multi-Task Tokens in Transformers for Skin Lesion Segmentation and Classification
Jingye Chen, Jieneng Chen, Zongwei Zhou +3
Recent advances in automated skin cancer diagnosis have yielded performance on par with board-certified dermatologists. However, these approaches formulated skin cancer diagnosis a…
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Nan Du, Yanping Huang, Andrew M. Dai +24
Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. For example, thanks to scaling, GPT-3 was able to ach…
Learning from Temporal Gradient for Semi-supervised Action Recognition
Junfei Xiao, Longlong Jing, Lin Zhang +5
Semi-supervised video action recognition tends to enable deep neural networks to achieve remarkable performance even with very limited labeled data. However, existing methods are m…
Seeking an Optimal Approach for Computer-Aided Pulmonary Embolism Detection
Nahid Ul Islam, Shiv Gehlot, Zongwei Zhou +2
Pulmonary embolism (PE) represents a thrombus ("blood clot"), usually originating from a lower extremity vein, that travels to the blood vessels in the lung, causing vascular obstr…
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
Yu Zhang, Daniel S. Park, Wei Han +23
We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a m…
Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration
Fatemeh Haghighi, Mohammad Reza Hosseinzadeh Taher, Zongwei Zhou +2
Medical images are naturally associated with rich semantics about the human anatomy, reflected in an abundance of recurring anatomical patterns, offering unique potential to foster…
UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh +1
The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations: (…
Scale MLPerf-0.6 models on Google TPU-v3 Pods
Sameer Kumar, Victor Bitorff, Dehao Chen +9
The recent submission of Google TPU-v3 Pods to the industry wide MLPerf v0.6 training benchmark demonstrates the scalability of a suite of industry relevant ML models. MLPerf defin…