Unsupervised Pretraining Encourages Moderate-Sparseness

Jun Li, Wei Luo, Jian Yang +1

It is well known that direct training of deep neural networks will generally lead to poor results. A major progress in recent years is the invention of various pretraining methods…