Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter
Qinghao Hu, Shang Yang, Junxian Guo +7
The emergence of Large Language Models (LLMs) with strong reasoning capabilities marks a significant milestone, unlocking new frontiers in complex problem-solving. However, trainin…
cs.LG2025
Optimizing Mixture of Block Attention
Guangxuan Xiao, Junxian Guo, Kasra Mazaheri +1
Mixture of Block Attention (MoBA) (Lu et al., 2025) is a promising building block for efficiently processing long contexts in LLMs by enabling queries to sparsely attend to a small…
cs.LG2025
SparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsity
Samir Khaki, Xiuyu Li, Junxian Guo +7
Fine-tuning LLMs is both computationally and memory-intensive. While parameter-efficient fine-tuning methods, such as QLoRA and DoRA, reduce the number of trainable parameters and…