4 citations · 8 across the 31 of their papers we have counts for
16 papers · 1 filter
V-ITI: Mitigating Hallucinations in Multimodal Large Language Models via Visual Inference-Time Intervention
Nan Sun, Zhenyu Zhang, Xixun Lin +8
Multimodal Large Language Models (MLLMs) excel in numerous vision-language tasks yet suffer from hallucinations, producing content inconsistent with input visuals, that undermine r…
Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding
Yuchen Feng, Zhenyu Zhang, Naibin Gu +8
Multimodal large language models (MLLMs) have achieved remarkable progress on various vision-language tasks, yet their visual perception remains limited. Humans, in comparison, per…
A Survey on Parallel Reasoning
Ziqi Wang, Boye Niu, Zipeng Gao +10
With the increasing capabilities of Large Language Models (LLMs), parallel reasoning has emerged as a new inference paradigm that enhances reasoning robustness by concurrently expl…
MatryoshkaThinking: Recursive Test-Time Scaling Enables Efficient Reasoning
Hongwei Chen, Yishu Lei, Dan Zhang +10
Test-time scaling has emerged as a promising paradigm in language modeling, wherein additional computational resources are allocated during inference to enhance model performance.…
A2R: An Asymmetric Two-Stage Reasoning Framework for Parallel Reasoning
Ziqi Wang, Boye Niu, Zhongli Li +7
Recent Large Reasoning Models have achieved significant improvements in complex task-solving capabilities by allocating more computation at the inference stage with a "thinking lon…
ProxyAttn: Guided Sparse Attention via Representative Heads
Yixuan Wang, Huang He, Siqi Bao +4
The quadratic complexity of attention mechanisms limits the efficiency of Large Language Models (LLMs) on long-text tasks. Recently, methods that dynamically estimate block importa…
Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factuality
Junliang Li, Yucheng Wang, Yan Chen +5
Hallucination in large language models (LLMs) during long-form generation remains difficult to address under existing reinforcement learning from human feedback (RLHF) frameworks,…
Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
Naibin Gu, Zhenyu Zhang, Yuchen Feng +8
Mixture-of-Experts (MoE) models typically fix the number of activated experts at both training and inference. However, real-world deployments often face heterogeneous hardware,…
Semantic Energy: Detecting LLM Hallucination Beyond Entropy
Huan Ma, Jiadong Pan, Jing Liu +7
Large Language Models (LLMs) are being increasingly deployed in real-world applications, but they remain susceptible to hallucinations, which produce fluent yet incorrect responses…
BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation
Yuhao Wang, Ruiyang Ren, Yucheng Wang +4
With the rapid advancement of large language models (LLMs), retrieval-augmented generation (RAG) has emerged as a critical approach to supplement the inherent knowledge limitations…
Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation
Yuhao Wang, Ruiyang Ren, Yucheng Wang +4
Considering the inherent limitations of parametric knowledge in large language models (LLMs), retrieval-augmented generation (RAG) is widely employed to expand their knowledge scop…
Advantageous Parameter Expansion Training Makes Better Large Language Models
Naibin Gu, Yilong Chen, Zhenyu Zhang +7
Although scaling up the number of trainable parameters can effectively improve the training performance of large language models, it also leads to increased computational overhead.…
Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation
Yuhao Wang, Ruiyang Ren, Yucheng Wang +4
Long-form question answering (LFQA) requires open-ended long-form responses that synthesize coherent, factually grounded content from multi-source evidence. This makes reinforcemen…
Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
Yilong Chen, Junyuan Shang, Zhenyu Zhang +7
Large language models (LLMs) face inherent performance bottlenecks under parameter constraints, particularly in processing critical tokens that demand complex reasoning. Empirical…
BeamLoRA: Beam-Constraint Low-Rank Adaptation
Naibin Gu, Zhenyu Zhang, Xiyu Liu +7
Due to the demand for efficient fine-tuning of large language models, Low-Rank Adaptation (LoRA) has been widely adopted as one of the most effective parameter-efficient fine-tunin…
Curiosity-Driven Reinforcement Learning from Human Feedback
Haoran Sun, Yekun Chai, Shuohuan Wang +3
Reinforcement learning from human feedback (RLHF) has proven effective in aligning large language models (LLMs) with human preferences, but often at the cost of reduced output dive…