How Could Polyhedral Theory Harness Deep Learning?
arXiv:1806.06365
Abstract
The holy grail of deep learning is to come up with an automatic method to design optimal architectures for different applications. In other words, how can we effectively dimension and organize neurons along the network layers based on the computational resources, input size, and amount of training data? We outline promising research directions based on polyhedral theory and mixed-integer representability that may offer an analytical approach to this question, in contrast to the empirical techniques often employed.
References in corpus (5)
- On the Number of Linear Regions of Deep Neural Networks
- Understanding Deep Neural Networks with Rectified Linear Units
- On the number of response regions of deep feed forward networks with piece-wise linear activations
- Deep Neural Networks as 0-1 Mixed Integer Linear Programs: A Feasibility Study
- Bounding and Counting Linear Regions of Deep Neural Networks