4 citations · 8 across the 31 of their papers we have counts for
32 papers
Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs
Zixuan Ren, Jinliang Lu, Junhong Wu +5
Model merging plays a crucial role in consolidating multiple specialized models into a single, unified model, especially in the era of large language models (LLMs). Recent research…
Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping
Yao Chen, Yilong Chen, Yinqi Yang +9
Existing approaches to increasing the effective depth of Transformers predominantly rely on parameter reuse, extending computation through recursive execution. Under this paradigm,…
Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models
Jiadong Pan, Liang Li, Yuxin Peng +6
Recently, unified multimodal models (UMMs) have made remarkable progress in integrating visual understanding and generation, demonstrating strong potential for complex text-to-imag…
Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation
Yilong Chen, Naibin Gu, Junyuan Shang +8
Mixture-of-Experts (MoE) decouples model capacity from per-token computation, yet their scalability remains limited by the physical dimensions of depth and width. To overcome this,…
Eureka-Audio: Triggering Audio Intelligence in Compact Language Models
Dan Zhang, Yishu Lei, Jing Hu +10
We present Eureka-Audio, a compact yet high-performance audio language model that achieves competitive performance against models that are 4 to 18 times larger across a broad range…
OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
Le Zhang, Yixiong Xiao, Xinjiang Lu +12
Graphical User Interface (GUI) agents show great potential for enabling foundation models to complete real-world tasks, revolutionizing human-computer interaction and improving hum…
CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillation
Jing Hu, Danxiang Zhu, Xianlong Luo +9
Large Audio Language Models (LALMs) have garnered significant research interest. Despite being built upon text-based large language models (LLMs), LALMs frequently exhibit a degrad…
VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction
Longbin Ji, Xiaoxiong Liu, Junyuan Shang +4
Recent advances in video generation have been dominated by diffusion and flow-matching models, which produce high-quality results but remain computationally intensive and difficult…
Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models
Shaoning Sun, Mingzhu Cai, Huang He +5
Language model families exhibit striking disparity in their capacity to benefit from reinforcement learning: under identical training, models like Qwen achieve substantial gains, w…
MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Free
Yishu Lei, Shuwei He, Jing Hu +9
Extending the input modality of Large Language Models~(LLMs) to the audio domain is essential for achieving comprehensive multimodal perception. However, it is well-known that acou…
MEnvAgent: Scalable Polyglot Environment Construction for Verifiable Software Engineering
Chuanzhe Guo, Jingjing Wu, Sijun He +10
The evolution of Large Language Model (LLM) agents for software engineering (SWE) is constrained by the scarcity of verifiable datasets, a bottleneck stemming from the complexity o…
V-ITI: Mitigating Hallucinations in Multimodal Large Language Models via Visual Inference-Time Intervention
Nan Sun, Zhenyu Zhang, Xixun Lin +8
Multimodal Large Language Models (MLLMs) excel in numerous vision-language tasks yet suffer from hallucinations, producing content inconsistent with input visuals, that undermine r…
Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding
Yuchen Feng, Zhenyu Zhang, Naibin Gu +8
Multimodal large language models (MLLMs) have achieved remarkable progress on various vision-language tasks, yet their visual perception remains limited. Humans, in comparison, per…
A Survey on Parallel Reasoning
Ziqi Wang, Boye Niu, Zipeng Gao +10
With the increasing capabilities of Large Language Models (LLMs), parallel reasoning has emerged as a new inference paradigm that enhances reasoning robustness by concurrently expl…
MatryoshkaThinking: Recursive Test-Time Scaling Enables Efficient Reasoning
Hongwei Chen, Yishu Lei, Dan Zhang +10
Test-time scaling has emerged as a promising paradigm in language modeling, wherein additional computational resources are allocated during inference to enhance model performance.…
A2R: An Asymmetric Two-Stage Reasoning Framework for Parallel Reasoning
Ziqi Wang, Boye Niu, Zhongli Li +7
Recent Large Reasoning Models have achieved significant improvements in complex task-solving capabilities by allocating more computation at the inference stage with a "thinking lon…
ProxyAttn: Guided Sparse Attention via Representative Heads
Yixuan Wang, Huang He, Siqi Bao +4
The quadratic complexity of attention mechanisms limits the efficiency of Large Language Models (LLMs) on long-text tasks. Recently, methods that dynamically estimate block importa…
Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factuality
Junliang Li, Yucheng Wang, Yan Chen +5
Hallucination in large language models (LLMs) during long-form generation remains difficult to address under existing reinforcement learning from human feedback (RLHF) frameworks,…
Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
Naibin Gu, Zhenyu Zhang, Yuchen Feng +8
Mixture-of-Experts (MoE) models typically fix the number of activated experts at both training and inference. However, real-world deployments often face heterogeneous hardware,…
Semantic Energy: Detecting LLM Hallucination Beyond Entropy
Huan Ma, Jiadong Pan, Jing Liu +7
Large Language Models (LLMs) are being increasingly deployed in real-world applications, but they remain susceptible to hallucinations, which produce fluent yet incorrect responses…
BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation
Yuhao Wang, Ruiyang Ren, Yucheng Wang +4
With the rapid advancement of large language models (LLMs), retrieval-augmented generation (RAG) has emerged as a critical approach to supplement the inherent knowledge limitations…
Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation
Yuhao Wang, Ruiyang Ren, Yucheng Wang +4
Considering the inherent limitations of parametric knowledge in large language models (LLMs), retrieval-augmented generation (RAG) is widely employed to expand their knowledge scop…
Advantageous Parameter Expansion Training Makes Better Large Language Models
Naibin Gu, Yilong Chen, Zhenyu Zhang +7
Although scaling up the number of trainable parameters can effectively improve the training performance of large language models, it also leads to increased computational overhead.…
Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation
Yuhao Wang, Ruiyang Ren, Yucheng Wang +4
Long-form question answering (LFQA) requires open-ended long-form responses that synthesize coherent, factually grounded content from multi-source evidence. This makes reinforcemen…