149 citations · 311 across the 8 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Mixture-of-Depths Attention
Lianghui Zhu, Yuxin Fang, Bencheng Liao +10
Scaling depth is a key driver for large language models (LLMs). Yet, as LLMs become deeper, they often suffer from signal degradation: informative features formed in shallow layers…
cs.CL2023★ 110 cited
Qwen Technical Report
Jinze Bai, Shuai Bai, Yunfei Chu +45
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to hu…