Custom Gradient Estimators are Straight-Through Estimators in Disguise

Matt Schoenbauer, Daniele Moro, Lukasz Lew +1

Quantization-aware training comes with a fundamental challenge: the derivative of quantization functions such as rounding are zero almost everywhere and nonexistent elsewhere. Vari…