Mixing Complexity and its Applications to Neural Networks

Michal Moshkovitz, Naftali Tishby

We suggest analyzing neural networks through the prism of space constraints. We observe that most training algorithms applied in practice use bounded memory, which enables us to us…