Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference

Bartosz Wójcik, Alessio Devoto, Karol Pustelnik +2

While transformer models have been highly successful, they are computationally inefficient. We observe that for each layer, the full width of the layer may be needed only for a sma…