From the 1 of 7 linked papers with an AI index.
7 papers
MoE-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation
Qingyu Yang, Haonan He, Minglei Li +4
Mixture-of-Experts (MoE) architectures have been widely adopted in large language models, yet parameter-efficient fine-tuning (PEFT) for MoE models remains underexplored. Existing…
Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent
Lei Bai, Zongsheng Cao, Yang Chen +50
The paper introduces Agents-A1, a 35B mixture-of-experts agent model that attains trillion-parameter-level performance by extending the length of reasoning horizons and integrating…
Parametric Skills
Xuan Zhao, Haonan He, Qingyu Yang +5
Since intelligence fundamentally relies on efficient skill acquisition (Chollet, 2019), the ability to leverage skills is critical. For LLMs, skills, manually authored or extracted…
Can Multimodal Large Language Models Truly Understand Small Objects?
Fujun Han, Junan Chen, Xintong Zhu +4
Multimodal Large Language Models (MLLMs) have shown promising potential in diverse understanding tasks, e.g., image and video analysis, math and physics olympiads. However, they re…
Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scale
Shengji Tang, Weihao Lin, Peng Ye +9
Large Language Models (LLMs) have rapidly advanced, with Gemini-3-Pro setting a new performance milestone. In this work, we explore collective intelligence as an alternative to mon…
A Unified Study of LoRA Variants: Taxonomy, Review, Codebase, and Empirical Evaluation
Haonan He, Jingqi Ye, Minglei Li +4
Low-Rank Adaptation (LoRA) is a fundamental parameter-efficient fine-tuning method that balances efficiency and performance in large-scale neural networks. However, the proliferati…