Orthogonalized SGD and Nested Architectures for Anytime Neural Networks

Chengcheng Wan, Henry Hoffmann, Shan Lu +1

We propose a novel variant of SGD customized for training network architectures that support anytime behavior: such networks produce a series of increasingly accurate outputs over…