MOYU: A Theoretical Study on Massive Over-activation Yielded Uplifts in LLMs

Chi Ma, Mincong Huang, Chao Wang +2

Massive Over-activation Yielded Uplifts(MOYU) is an inherent property of large language models, and dynamic activation(DA) based on the MOYU property is a clever yet under-explored…