4 citations · 18 across the 33 of their papers we have counts for
28 papers · 1 filter
Self-Distillation with Verbal Regularization
Jiazheng Li, Yawei Li, Junru Lu +7
On-policy self-distillation (OPSD) trains a large language model using supervision from a teacher instantiated from the same model as the student. The teacher is conditioned on pri…
Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens
Yizhen Yao, Qinglin Zhu, Runcong Zhao +4
Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategie…
Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention
Siya Qi, Yudong Chen, Runcong Zhao +6
Hallucination detection is critical for ensuring the reliability of large language models (LLMs) in context-based generation. Prior work has explored intrinsic signals available du…
Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding
Yanzheng Xiang, Lan Wei, Yizhen Yao +8
Parallel diffusion decoding can accelerate diffusion language model inference by unmasking multiple tokens per step, but aggressive parallelism often harms quality. Revocable decod…
Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation
Zhanghao Hu, Qinglin Zhu, Runcong Zhao +4
Standard Retrieval Augmented Generation (RAG) is poorly matched to agent memory. Unlike large heterogeneous corpora, agent memory forms a bounded and coherent interaction stream in…
SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
Zhenyi Shen, Junru Lu, Lin Gui +4
Sparse attention reduces the quadratic complexity of full self-attention but faces two challenges: (1) an attention gap, where applying sparse attention to full-attention-trained m…
Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States
Qinglin Zhu, Yizhen Yao, Runcong Zhao +7
Autoregressive (AR) models remain the standard for natural language generation but still suffer from high latency due to strictly sequential decoding. Recent diffusion-inspired app…
Beyond Perplexity: Let the Reader Select Retrieval Summaries via Spectrum Projection Score
Zhanghao Hu, Qinglin Zhu, Siya Qi +3
Large Language Models (LLMs) have shown improved generation performance through retrieval-augmented generation (RAG) following the retriever-reader paradigm, which supplements mode…
SymbolicThought: Integrating Language Models and Symbolic Reasoning for Consistent and Interpretable Human Relationship Understanding
Runcong Zhao, Qinglin Zhu, Hainiu Xu +3
Understanding character relationships is essential for interpreting complex narratives and conducting socially grounded AI research. However, manual annotation is time-consuming an…
Sparse Activation Editing for Reliable Instruction Following in Narratives
Runcong Zhao, Chengyu Cao, Qinglin Zhu +5
Complex narrative contexts often challenge language models' ability to follow instructions, and existing benchmarks fail to capture these difficulties. To address this, we propose…
Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration
Qinglin Zhu, Runcong Zhao, Hanqi Yan +3
Large Language Models (LLMs) struggle with complex reasoning due to limited diversity and inefficient search. We propose Soft Reasoning, an embedding-based search framework that op…
SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers
Yanzheng Xiang, Hanqi Yan, Shuyin Ouyang +2
This study evaluates large language models (LLMs) in generating code from algorithm descriptions in recent NLP papers. The task requires two key competencies: (1) algorithm compreh…
Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering
Zhanghao Hu, Hanqi Yan, Qinglin Zhu +3
Large language models have recently pushed open domain question answering (ODQA) to new frontiers. However, prevailing retriever-reader pipelines often depend on multiple rounds of…
RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following
Junru Lu, Jiazheng Li, Guodong Shen +5
Role-playing is important for Large Language Models (LLMs) to follow diverse instructions while maintaining role identity and the role's pre-defined ability limits. Existing role-p…
Two Heads Are Better Than One: Dual-Model Verbal Reflection at Inference-Time
Jiazheng Li, Yuxiang Zhou, Junru Lu +4
Although preference optimization methods have improved reasoning performance in Large Language Models (LLMs), they often lack transparency regarding why one reasoning outcome is pr…
PECAN: LLM-Guided Dynamic Progress Control with Attention-Guided Hierarchical Weighted Graph for Long-Document QA
Xinyu Wang, Yanzheng Xiang, Lin Gui +1
Long-document QA presents challenges with large-scale text and long-distance dependencies. Recent advances in Large Language Models (LLMs) enable entire documents to be processed i…
Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems
Italo Luis da Silva, Hanqi Yan, Lin Gui +1
The inherent ambiguity of cause and effect boundaries poses a challenge in evaluating causal event extraction tasks. Traditional metrics like Exact Match and BertScore poorly refle…
Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective
Hanqi Yan, Yanzheng Xiang, Guangyi Chen +3
To better interpret the intrinsic mechanism of large language models (LLMs), recent studies focus on monosemanticity on its basic units. A monosemantic neuron is dedicated to a sin…
PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games
Qinglin Zhu, Runcong Zhao, Bin Liang +3
We introduce WellPlay, a reasoning dataset for multi-agent conversational inference in Murder Mystery Games (MMGs). WellPlay comprises 1,482 inferential questions across 12 games,…
Large Language Models Fall Short: Understanding Complex Relationships in Detective Narratives
Runcong Zhao, Qinglin Zhu, Hainiu Xu +4
Existing datasets for narrative understanding often fail to represent the complexity and uncertainty of relationships in real-life social scenarios. To address this gap, we introdu…
Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Models
Yanzheng Xiang, Hanqi Yan, Lin Gui +1
In-context learning has become a popular paradigm in natural language processing. However, its performance can be significantly influenced by the order of in-context demonstration…
Mirror: A Multiple-perspective Self-Reflection Method for Knowledge-rich Reasoning
Hanqi Yan, Qinglin Zhu, Xinyu Wang +2
While Large language models (LLMs) have the capability to iteratively reflect on their own outputs, recent studies have observed their struggles with knowledge-rich problems withou…
Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond
Xinyu Wang, Hainiu Xu, Lin Gui +1
Task embedding, a meta-learning technique that captures task-specific information, has gained popularity, especially in areas such as multi-task learning, model editing, and interp…
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis
Yuxiang Zhou, Jiazheng Li, Yanzheng Xiang +3
Understanding in-context learning (ICL) capability that enables large language models (LLMs) to excel in proficiency through demonstration examples is of utmost importance. This im…