189 citations · 453 across the 62 of their papers we have counts for
10 papers · 1 filter
JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
Bingxiang He, Zekai Qu, Zeyuan Liu +9
Recent advances in reinforcement learning for large language models have converged on increasing complexity: multi-stage training pipelines, dynamic hyperparameter schedules, and c…
H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
Cheng Gao, Huimin Chen, Chaojun Xiao +3
Large language models (LLMs) frequently generate hallucinations -- plausible but factually incorrect outputs -- undermining their reliability. While prior work has examined halluci…
Large Language Models' Complicit Responses to Illicit Instructions across Socio-Legal Contexts
Xing Wang, Huiyuan Xie, Yiyan Wang +7
Large language models (LLMs) are now deployed at unprecedented scale, assisting millions of users in daily tasks. However, the risk of these models assisting unlawful activities re…
CoPRIS: Efficient and Stable Reinforcement Learning via Concurrency-Controlled Partial Rollout with Importance Sampling
Zekai Qu, Yinxu Pan, Ao Sun +2
Reinforcement learning (RL) post-training has become a trending paradigm for enhancing the capabilities of large language models (LLMs). Most existing RL systems for LLMs operate i…
NOSA: Native and Offloadable Sparse Attention
Yuxiang Huang, Pengjie Wang, Jicheng Han +9
Decoding throughput improvements from larger inference batches are limited by GPU memory, which is largely consumed by the key-value (KV) cache. Prior training-free KV cache offloa…
InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
Weilin Zhao, Zihan Zhou, Zhou Su +10
Long-sequence processing is a critical capability for modern large language models. However, the self-attention mechanism in the standard Transformer architecture faces severe comp…
BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
Chenyang Song, Weilin Zhao, Xu Han +5
To alleviate the computational burden of large language models (LLMs), architectures with activation sparsity, represented by mixture-of-experts (MoE), have attracted increasing at…
Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets Retrieval Augmented Generation Across Learning Style
Debdeep Sanyal, Agniva Maiti, Umakanta Maharana +4
Effective teaching requires adapting instructional strategies to accommodate the diverse cognitive and behavioral profiles of students, a persistent challenge in education and teac…
Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Data
Yudong Wang, Zixuan Fu, Jie Cai +9
Data quality has become a key factor in enhancing model performance with the rapid development of large language models (LLMs). Model-driven data filtering has increasingly become…
APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
Yuxiang Huang, Mingye Li, Xu Han +7
While long-context inference is crucial for advancing large language model (LLM) applications, its prefill speed remains a significant bottleneck. Current approaches, including seq…