Knowledge Transfer Pre-training

Zhiyuan Tang, Dong Wang, Yiqiao Pan +1

Pre-training is crucial for learning deep neural networks. Most of existing pre-training methods train simple models (e.g., restricted Boltzmann machines) and then stack them layer…