The Hidden Power of Pure 16-bit Floating-Point Neural Networks

Juyoung Yun, Byungkon Kang, Zhoulai Fu

Lowering the precision of neural networks from the prevalent 32-bit precision has long been considered harmful to performance, despite the gain in space and time. Many works propos…