2 citations · 8 across the 13 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024
When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
Haonan Wang, Qian Liu, Chao Du +4
Extending context window sizes allows large language models (LLMs) to process longer sequences and handle more complex tasks. Rotary Positional Embedding (RoPE) has become the de f…
cs.CL2024★ 1 cited
Investigating Layer Importance in Large Language Models
Yang Zhang, Yanfei Dong, Kenji Kawaguchi
Large language models (LLMs) have gained increasing attention due to their prominent ability to understand and process texts. Nevertheless, LLMs largely remain opaque. The lack of…