138 citations · 294 across the 13 of their papers we have counts for
1 paper · 2 filters
Disen Lan, Jianbin Zheng, Yuxi Ren +5
Hybrid attention models improve long-context efficiency by retaining only a subset of full-attention layers and replacing the remaining layers with linear attention. However, the e…