Zero-bias autoencoders and the benefits of co-adapting features

Kishore Konda, Roland Memisevic, David Krueger

Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidd…