6 citations · 12 across the 24 of their papers we have counts for
4 papers · 2 filters
Beyond State Capacity: Why Retrieval Breaks in Mamba
Nir Yarden, Amit Ben-Artzy, Roy Schwartz
Mamba models underperform Transformers on in-context retrieval. This gap is usually attributed to Mamba's fixed-size recurrent state, which loses information as the context grows.…
From Knowledge Access to Source Learning: Developing Source-Specific Competence
Lucheng Fu, Kejing Xia, Yiyang Wang +10
Large language model (LLM) agents increasingly rely on persistent external sources to solve sequences of knowledge-intensive tasks. Existing methods improve how source content is a…
UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
Yiqiao Jin, Yiyang Wang, Lucheng Fu +7
Self-distillation (SD) offers a promising path for adapting large language models (LLMs) without relying on stronger external teachers. However, SD in autoregressive LLMs remains c…
TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization
Lucheng Fu, Ye Yu, Yiyang Wang +4
Large language models (LLMs) are highly sensitive to the prompts used to specify task objectives and behavioral constraints. Many recent prompt optimization methods iteratively rew…