144 citations · 391 across the 88 of their papers we have counts for
34 papers · 1 filter
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
Shicheng Xu, Liang Pang, Liyi Chen +7
Retrieval-augmented generation (RAG) improves factuality but adds latency and engineering overhead at serving time. We propose RING (Retrieval-Internalized Generation), a holistic…
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Zihao Wei, Wenjie Shi, Liang Pang +8
Long-form chain-of-thought reasoning can improve LLM performance on complex tasks, but models often continue generating unnecessary reasoning after a correct answer has emerged. We…
D-Models and E-Models: Diversity-Stability Trade-offs in the Sampling Behavior of Large Language Models
Jia Gu, Liang Pang, Huawei Shen +1
The predictive probability of the next token (P_token) in large language models (LLMs) is inextricably linked to the probability of relevance for the next piece of information, the…
LLM Latent Reasoning as Chain of Superposition
Jingcheng Deng, Liang Pang, Zihao Wei +6
Latent reasoning offers a computation-efficient alternative to Chain-of-Thought but often suffers from performance degradation due to distributional misalignment and ambiguous chai…
Large Language Model Sourcing: A Survey
Liang Pang, Jia Gu, Sunhao Dai +7
Due to the black-box nature of large language models (LLMs) and the realism of their generated content, issues such as hallucinations, bias, unfairness, and copyright infringement…
Fine-tuning Done Right in Model Editing
Wanli Yang, Rui Tang, Hongyu Zang +6
Fine-tuning, a foundational method for adapting large language models, has long been considered ineffective for model editing. Here, we challenge this belief, arguing that the repo…
The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis
Zihao Wei, Liang Pang, Jiahao Liu +7
Test-time scaling via explicit reasoning trajectories significantly boosts large language model (LLM) performance but often triggers overthinking. To explore this, we analyze reaso…
GloSS over Toxicity: Understanding and Mitigating Toxicity in LLMs via Global Toxic Subspace
Zenghao Duan, Zhiyi Yin, Zhichao Shi +6
This paper investigates the underlying mechanisms of toxicity generation in Large Language Models (LLMs) and proposes an effective detoxification approach. Prior work typically con…
Inference-time Alignment in Continuous Space
Yige Yuan, Teng Xiao, Li Yunfan +5
Aligning large language models with human feedback at inference time has received increasing attention due to its flexibility. Existing methods rely on generating multiple response…
Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs
Hexiang Tan, Fei Sun, Sha Liu +8
As large language models (LLMs) often generate plausible but incorrect content, error detection has become increasingly critical to ensure truthfulness. However, existing detection…
RLKD: Distilling LLMs' Reasoning via Reinforcement Learning
Shicheng Xu, Liang Pang, Yunchang Zhu +6
Distilling reasoning paths from teacher to student models via supervised fine-tuning (SFT) provides a shortcut for improving the reasoning ability of smaller Large Language Models…
Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Models
Yilong Xu, Jinhua Gao, Xiaoming Yu +4
Retrieval-Augmented Language Models boost task performance, owing to the retriever that provides external knowledge. Although crucial, the retriever primarily focuses on semantics…
Related Knowledge Perturbation Matters: Rethinking Multiple Pieces of Knowledge Editing in Same-Subject
Zenghao Duan, Wenbin Duan, Zhiyi Yin +5
Knowledge editing has become a promising approach for efficiently and precisely updating knowledge embedded in large language models (LLMs). In this work, we focus on Same-Subject…
Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment
Jingcheng Deng, Zhongtao Jiang, Liang Pang +5
A new trend uses LLMs as dense text encoders via contrastive learning. However, since LLM embeddings predict the probability distribution of the next token, they are inherently gen…
On the Diminishing Returns of Complex Robust RAG Training in the Era of Powerful LLMs
Hanxing Ding, Shuchang Tao, Liang Pang +5
Retrieval-augmented generation (RAG) systems traditionally employ sophisticated training strategies to enhance robustness against retrieval noise. In this work, we investigate a cr…
The Mirage of Model Editing: Revisiting Evaluation in the Wild
Wanli Yang, Fei Sun, Jiajun Tan +5
Despite near-perfect results reported in the literature, the effectiveness of model editing in real-world applications remains unclear. To bridge this gap, we introduce QAEdit, a n…
Fact-Level Confidence Calibration and Self-Correction
Yige Yuan, Bingbing Xu, Hexiang Tan +5
Confidence calibration in LLMs, i.e., aligning their self-assessed confidence with the actual accuracy of their responses, enabling them to self-evaluate the correctness of their o…
Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration
Kangxi Wu, Liang Pang, Huawei Shen +1
The black-box nature of large language models (LLMs) poses challenges in interpreting results, impacting issues such as data intellectual property protection and hallucination trac…
ALiiCE: Evaluating Positional Fine-grained Citation Generation
Yilong Xu, Jinhua Gao, Xiaoming Yu +3
Large Language Model (LLM) can enhance its credibility and verifiability by generating text with citations. However, existing research on citation generation is predominantly limit…
A Theory for Token-Level Harmonization in Retrieval-Augmented Generation
Shicheng Xu, Liang Pang, Huawei Shen +1
Retrieval-augmented generation (RAG) utilizes retrieved texts to enhance large language models (LLMs). Studies show that while RAG provides valuable external information (benefit),…
Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models
Jingcheng Deng, Zihao Wei, Liang Pang +3
Recent knowledge editing methods have primarily focused on modifying structured knowledge in large language models. However, this task setting overlooks the fact that a significant…
Knowledge Boundary and Persona Dynamic Shape A Better Social Media Agent
Junkai Zhou, Liang Pang, Ya Jing +3
Constructing personalized and anthropomorphic agents holds significant importance in the simulation of social networks. However, there are still two key problems in existing works:…
Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation
Jia Gu, Liang Pang, Huawei Shen +1
With the rapid advancement of large language models (LLMs) for handling complex language tasks, an increasing number of studies are employing LLMs as agents to emulate the sequenti…
MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models
Zihao Wei, Jingcheng Deng, Liang Pang +3
The extensive utilization of large language models (LLMs) underscores the crucial necessity for precise and contemporary knowledge embedded within their intrinsic parameters. Exist…