4 citations · 4 across the 5 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
cs.CL2025
SWAA: Sliding Window Attention Adaptation for Efficient and Quality Preserving Long Context Processing
Yijiong Yu, Jiale Liu, Qingyun Wu +2
The quadratic complexity of self attention in Transformer based LLMs renders long context inference prohibitively expensive. While Sliding Window Attention (SWA), the simplest spar…
cs.CL2025
OpenCSG Chinese Corpus: A Series of High-quality Chinese Datasets for LLM Training
Yijiong Yu, Ziyun Dai, Zekun Wang +3
Large language models (LLMs) have demonstrated remarkable capabilities, but their success heavily relies on the quality of pretraining corpora. For Chinese LLMs, the scarcity of hi…