6 papers
CU in RAG: Compositional Concept Unlearning in Retrieval-Augmented Generation
Yiyang Duan, Fan Wu, Tiantong Wu +5
Retrieval-augmented generation can expose unwanted concepts through retrieved evidence even when the generator is moderated. We study query-side concept suppression with a fixed en…
SkillFocus: Evolving Agent Skills via Capability Decomposition
Ning Wang, Zhiren Gong, Bingdong Li +2
Agent skill evolution seeks to improve reusable procedural guidance for large language model (LLM) agents through iterative revision. Existing methods base each revision mainly on…
State of Thought Enables Endogenous Reasoning
Zhiren Gong, Yikun Hou, Zihao Zeng +3
Test-time compute has emerged as a major approach to improving the capabilities of Large Language Models (LLMs). However, existing test-time reasoning paradigms rely heavily on ext…
CoCurve: Cross-Module Co-Pruning Curvature for Structured LLM Pruning
Zhiren Gong, Zihao Zeng, Zijie Wang +6
Resource-constrained deployment requires sustaining large language model (LLM) capabilities as models scale under fixed memory and computation budgets. Structured pruning advances…
Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits
Zhiren Gong, He Lu, Zihao Zeng +6
Mechanistic interpretability seeks to explain transformer behavior through circuits: sets of internal components that causally support a behavior. However, self-repair creates a bl…
XDomainBench: Diagnosing Reasoning Collapse in High-Dimensional Scientific Knowledge Composition
Gong Zhiren, Tiantong Wu, Jiaming Zhang +9
Large Language Models (LLMs) are increasingly deployed for knowledge synthesis, yet their capacity for compositional generalization in scientific knowledge remains under-characteri…