Climbing the WOL: Training for Cheaper Inference

Zichang Liu, Zhaozhuo Xu, Alan Ji +3

Efficient inference for wide output layers (WOLs) is an essential yet challenging task in large scale machine learning. Most approaches reduce this problem to approximate maximum i…