7 papers
Train What You Deploy: Closing the MLP Reachability Gap in Low-Rank Clone Distillation
Wenhui Chen, Zhifeng Li, Jie Zhou +5
A compressed student has two shapes that need not agree: the weight it deploys at inference and the weight family its training can reach. We show that a state-of-the-art weight-inh…
How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks
Yanlin Fei, Nazhou Liu, Xinmiao Yu +7
AI has long assisted scientific research, but the rapid advance of LLMs and agentic scaffolds is reshaping the landscape; a single system can now carry whole-stage research from an…
Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies
Shaolong Chen, Yanlin Fei, Nazhou Liu +7
Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? We introduce Reconstruction, a blind idea-recove…
Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing
Li Lei, Madalina Ciobanu, Qingqing Mao +1
LLMs increasingly require surgical model editing to enhance domain-specific capabilities without incurring the computational cost or catastrophic forgetting associated with full fi…
AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
Shaolong Chen, Madalina Ciobanu, Qingqing Mao +1
DPO has become a widely adopted alternative to RLHF for aligning LLMs with human preferences, eliminating the need for a separate reward model or RL loop. Recent theoretical analys…
State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation
Navan Preet Singh, Anurag Garikipati, Ahmed Abulkhair +6
This paper introduces Arabic-DeepSeek-R1, an application-driven open-source Arabic LLM that leverages a sparse MoE backbone to address the digital equity gap for under-represented…