Learned Token Pruning for Transformers

Sehoon Kim, Sheng Shen, David Thorsley +4

Deploying transformer models in practice is challenging due to their inference cost, which scales quadratically with input sequence length. To address this, we present a novel Lear…