45 citations · 114 across the 13 of their papers we have counts for
13 papers
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding
Haoli Bai, Zhiguang Liu, Xiaojun Meng +9
Unsupervised pre-training on millions of digital-born or scanned documents has shown promising advances in visual document understanding~(VDU). While various vision-language pre-tr…
Modern Distributed Data-Parallel Large-Scale Pre-training Strategies For NLP models
Hao Bai
Distributed deep learning is becoming increasingly popular due to the expanding demand for computing resources for deep learning models with a larger amount of parameters. Differen…
Dynamically pruning segformer for efficient semantic segmentation
Haoli Bai, Hongda Mao, Dinesh Nair
As one of the successful Transformer-based models in computer vision tasks, SegFormer demonstrates superior performance in semantic segmentation. Nevertheless, the high computation…
Towards Efficient Post-training Quantization of Pre-trained Language Models
Haoli Bai, Lu Hou, Lifeng Shang +3
Network quantization has gained increasing attention with the rapid growth of large pre-trained language models~(PLMs). However, most existing quantization methods for PLMs follow…
Discrete Auto-regressive Variational Attention Models for Text Modeling
Xianghong Fang, Haoli Bai, Jian Li +3
Variational autoencoders (VAEs) have been widely applied for text modeling. In practice, however, they are troubled by two challenges: information underrepresentation and posterior…
BinaryBERT: Pushing the Limit of BERT Quantization
Haoli Bai, Wei Zhang, Lu Hou +6
The rapid development of large pre-trained language models has greatly increased the demand for model compression techniques, among which quantization is a popular solution. In thi…
Discrete Variational Attention Models for Language Generation
Xianghong Fang, Haoli Bai, Zenglin Xu +2
Variational autoencoders have been widely applied for natural language generation, however, there are two long-standing problems: information under-representation and posterior col…
Efficient Bitwidth Search for Practical Mixed Precision Neural Network
Yuhang Li, Wei Wang, Haoli Bai +3
Network quantization has rapidly become one of the most widely used methods to compress and accelerate deep neural networks. Recent efforts propose to quantize weights and activati…
RTN: Reparameterized Ternary Network
Yuhang Li, Xin Dong, Sai Qian Zhang +3
To deploy deep neural networks on resource-limited devices, quantization has been widely explored. In this work, we study the extremely low-bit networks which have tremendous speed…
Few Shot Network Compression via Cross Distillation
Haoli Bai, Jiaxiang Wu, Irwin King +1
Model compression has been widely adopted to obtain light-weighted deep neural networks. Most prevalent methods, however, require fine-tuning with sufficient training data to ensur…
Structured Pruning of Recurrent Neural Networks through Neuron Selection
Liangjian Wen, Xuanyang Zhang, Haoli Bai +1
Recurrent neural networks (RNNs) have recently achieved remarkable successes in a number of applications. However, the huge sizes and computational burden of these models make it d…
DART: Domain-Adversarial Residual-Transfer Networks for Unsupervised Cross-Domain Image Classification
Xianghong Fang, Haoli Bai, Ziyi Guo +3
The accuracy of deep learning (e.g., convolutional neural networks) for an image classification task critically relies on the amount of labeled training data. Aiming to solve an im…
Stochastic Sequential Neural Networks with Structured Inference
Hao Liu, Haoli Bai, Lirong He +1
Unsupervised structure learning in high-dimensional time series data has attracted a lot of research interests. For example, segmenting and labelling high dimensional time series c…