From the 1 of 12 linked papers with an AI index.
13 papers
DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data
Zhen Huang, Yikun Wang, Shijie Xia +1
Pretraining data processing is critical to the downstream performance of Large Language Models (LLMs). However, many existing approaches define a fixed processing strategy at the c…
UMoE:Unlocking Every Expert in Domain-Specific Training
Xuefeng Li, Pengfei Liu
The paper introduces UMoE, a method that prunes low‑saliency experts and regrows new ones to better align a mixture‑of‑experts language model with a target domain before fine‑tunin…
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
Chongyu Fan, Pengfei Liu, Jingjia Huang +2
Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs). To achieve sample efficiency, modern…
Diagnosing and Mitigating Context Rot in Long-horizon Search
Shijie Xia, Yikun Wang, Zhen Huang +1
Extensive context has become the norm as Large Language Models (LLMs) are increasingly deployed in long-horizon search tasks. The concern that increasing context length degrades mo…
World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis
Yi Yang, Zhihong Liu, Siqi Kou +9
We propose world-language-action (WLA) models as a new class of embodied foundation models. WLA takes textual instructions, images, and robot states as inputs to jointly predict te…
AcademiClaw: When Students Set Challenges for AI Agents
Junjie Yu, Pengrui Lu, Weiye Si +75
Benchmarks within the OpenClaw ecosystem have thus far evaluated exclusively assistant-level tasks, leaving the academic-level capabilities of OpenClaw largely unexamined. We intro…