7 citations · 29 across the 54 of their papers we have counts for
28 papers · 1 filter
ZenGen: Social Mind for LLMs
ZenGen Team, Ao Xiang, Bi Jingping +57
As large language models move from isolated task solving toward long-term service in human environments, they require social intelligence: the ability to infer mental states, track…
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
Shicheng Xu, Liang Pang, Liyi Chen +7
Retrieval-augmented generation (RAG) improves factuality but adds latency and engineering overhead at serving time. We propose RING (Retrieval-Internalized Generation), a holistic…
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Zihao Wei, Wenjie Shi, Liang Pang +8
Long-form chain-of-thought reasoning can improve LLM performance on complex tasks, but models often continue generating unnecessary reasoning after a correct answer has emerged. We…
D-Models and E-Models: Diversity-Stability Trade-offs in the Sampling Behavior of Large Language Models
Jia Gu, Liang Pang, Huawei Shen +1
The predictive probability of the next token (P_token) in large language models (LLMs) is inextricably linked to the probability of relevance for the next piece of information, the…
LLM Latent Reasoning as Chain of Superposition
Jingcheng Deng, Liang Pang, Zihao Wei +6
Latent reasoning offers a computation-efficient alternative to Chain-of-Thought but often suffers from performance degradation due to distributional misalignment and ambiguous chai…
Large Language Model Sourcing: A Survey
Liang Pang, Jia Gu, Sunhao Dai +7
Due to the black-box nature of large language models (LLMs) and the realism of their generated content, issues such as hallucinations, bias, unfairness, and copyright infringement…
The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis
Zihao Wei, Liang Pang, Jiahao Liu +7
Test-time scaling via explicit reasoning trajectories significantly boosts large language model (LLM) performance but often triggers overthinking. To explore this, we analyze reaso…
GloSS over Toxicity: Understanding and Mitigating Toxicity in LLMs via Global Toxic Subspace
Zenghao Duan, Zhiyi Yin, Zhichao Shi +6
This paper investigates the underlying mechanisms of toxicity generation in Large Language Models (LLMs) and proposes an effective detoxification approach. Prior work typically con…
RLKD: Distilling LLMs' Reasoning via Reinforcement Learning
Shicheng Xu, Liang Pang, Yunchang Zhu +6
Distilling reasoning paths from teacher to student models via supervised fine-tuning (SFT) provides a shortcut for improving the reasoning ability of smaller Large Language Models…
ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models
Hanxing Ding, Shuchang Tao, Liang Pang +5
Tool learning has emerged as a crucial capability for large language models (LLMs) to solve complex real-world tasks through interaction with external tools. Existing approaches fa…
Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment
Jingcheng Deng, Zhongtao Jiang, Liang Pang +5
A new trend uses LLMs as dense text encoders via contrastive learning. However, since LLM embeddings predict the probability distribution of the next token, they are inherently gen…
On the Diminishing Returns of Complex Robust RAG Training in the Era of Powerful LLMs
Hanxing Ding, Shuchang Tao, Liang Pang +5
Retrieval-augmented generation (RAG) systems traditionally employ sophisticated training strategies to enhance robustness against retrieval noise. In this work, we investigate a cr…
Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration
Kangxi Wu, Liang Pang, Huawei Shen +1
The black-box nature of large language models (LLMs) poses challenges in interpreting results, impacting issues such as data intellectual property protection and hallucination trac…
A Theory for Token-Level Harmonization in Retrieval-Augmented Generation
Shicheng Xu, Liang Pang, Huawei Shen +1
Retrieval-augmented generation (RAG) utilizes retrieved texts to enhance large language models (LLMs). Studies show that while RAG provides valuable external information (benefit),…
Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models
Jingcheng Deng, Zihao Wei, Liang Pang +3
Recent knowledge editing methods have primarily focused on modifying structured knowledge in large language models. However, this task setting overlooks the fact that a significant…
Knowledge Boundary and Persona Dynamic Shape A Better Social Media Agent
Junkai Zhou, Liang Pang, Ya Jing +3
Constructing personalized and anthropomorphic agents holds significant importance in the simulation of social networks. However, there are still two key problems in existing works:…
Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation
Jia Gu, Liang Pang, Huawei Shen +1
With the rapid advancement of large language models (LLMs) for handling complex language tasks, an increasing number of studies are employing LLMs as agents to emulate the sequenti…
MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models
Zihao Wei, Jingcheng Deng, Liang Pang +3
The extensive utilization of large language models (LLMs) underscores the crucial necessity for precise and contemporary knowledge embedded within their intrinsic parameters. Exist…
Qsnail: A Questionnaire Dataset for Sequential Question Generation
Yan Lei, Liang Pang, Yuanzhuo Wang +2
The questionnaire is a professional research methodology used for both qualitative and quantitative analysis of human opinions, preferences, attitudes, and behaviors. However, desi…
Stable Knowledge Editing in Large Language Models
Zihao Wei, Liang Pang, Hanxing Ding +3
Efficient knowledge editing of large language models is crucial for replacing obsolete information or incorporating specialized knowledge on a large scale. However, previous method…
Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation
Shicheng Xu, Liang Pang, Mo Yu +4
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating additional information from retrieval. However, studies have shown that LLMs still face…
Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs
Hanxing Ding, Liang Pang, Zihao Wei +2
Hallucinations present a significant challenge for large language models (LLMs). The utilization of parametric knowledge in generating factual content is constrained by the limited…
Think Before You Speak: Cultivating Communication Skills of Large Language Models via Inner Monologue
Junkai Zhou, Liang Pang, Huawei Shen +1
The emergence of large language models (LLMs) further improves the capabilities of open-domain dialogue systems and can generate fluent, coherent, and diverse responses. However, L…
RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling
Jingcheng Deng, Liang Pang, Huawei Shen +1
Retrieval-augmented language models show promise in addressing issues like outdated information and hallucinations in language models (LMs). However, current research faces two mai…