Triangular Dropout: Variable Network Width without Retraining

Edward W. Staley, Jared Markowitz

One of the most fundamental design choices in neural networks is layer width: it affects the capacity of what a network can learn and determines the complexity of the solution. Thi…