11 papers
LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning
Yu Zhao, Zekun Zhang, Fan Jiang +6
Recent advances in long chain-of-thought reasoning models such as DeepSeek-R1 have led to increasingly longer inference context lengths under the test-time scaling paradigm. Howeve…
CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks
Peiqin Lin, Chenyang Lyu, Wenjiang Luo +22
Large language models (LLMs) are now deployed worldwide, inspiring a surge of benchmarks that measure their multilingual and multicultural abilities. However, these benchmarks prio…
Difficulty-Estimated Policy Optimization
Yu Zhao, Fan Jiang, Tianle Liu +4
Recent advancements in Large Reasoning Models (LRMs), exemplified by DeepSeek-R1, have underscored the potential of scaling inference-time compute through Group Relative Policy Opt…
EgoLCD: Egocentric Video Generation with Long Context Diffusion
Liuzhou Zhang, Jiarui Ye, Yuanlei Wang +6
Generating long, coherent egocentric videos is difficult, as hand-object interactions and procedural tasks require reliable long-term memory. Existing autoregressive models suffer…
RODS: Robust Optimization Inspired Diffusion Sampling for Detecting and Reducing Hallucination in Generative Models
Yiqi Tian, Pengfei Jin, Mingze Yuan +3
Diffusion models have achieved state-of-the-art performance in generative modeling, yet their sampling procedures remain vulnerable to hallucinations-often stemming from inaccuraci…
Long-Context Attention Benchmark: From Kernel Efficiency to Distributed Context Parallelism
Tao Bu, Qiangang Wang, Bowen Zeng +4
Transformer-based large language models (LLMs) have achieved remarkable success, yet their standard attention mechanism incurs quadratic computation and memory costs with respect t…