Learning to Skip for Language Modeling

Dewen Zeng, Nan Du, Tao Wang +4

Overparameterized large-scale language models have impressive generalization performance of in-context few-shot learning. However, most language models allocate the same amount of…