1.9k citations · 4.2k across the 30 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024★ 8 cited
Massive Activations in Large Language Models
Mingjie Sun, Xinlei Chen, J. Zico Kolter +1
We observe an empirical phenomenon in Large Language Models (LLMs) -- very few activations exhibit significantly larger values than others (e.g., 100,000 times larger). We call the…
cs.CL2023★ 53 cited
A Simple and Effective Pruning Approach for Large Language Models
Mingjie Sun, Zhuang Liu, Anna Bair +1
As their size increases, Large Languages Models (LLMs) are natural candidates for network pruning methods: approaches that drop a subset of network weights while striving to preser…