8 papers
MoE-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation
Qingyu Yang, Haonan He, Minglei Li +4
Mixture-of-Experts (MoE) architectures have been widely adopted in large language models, yet parameter-efficient fine-tuning (PEFT) for MoE models remains underexplored. Existing…
Learning from Execution: Self-Evolving Memory for Private-Library Code Generation
Mofei Li, Taozhi Chen, Guowei Yang +1
Large Language Models (LLMs) have achieved strong performance on general code generation, but their effectiveness drops sharply in enterprise settings where software development re…
DiBS: Diffusion-Informed Branch Selection
Bo Liu, Yuan Xie, Yuan Gao +4
Sudoku is a representative constraint satisfaction problem that requires global structural reasoning under strict discrete constraints. The existing works of solving Sudoku mainly…
FRISM: Fine-Grained Reasoning Injection via Subspace-Level Model Merging for Vision-Language Models
Chenyu Huang, Peng Ye, Xudong Tan +4
Efficiently enhancing the reasoning capabilities of Vision-Language Models (VLMs) by merging them with Large Reasoning Models (LRMs) has emerged as a promising direction. However,…
Can Multimodal Large Language Models Truly Understand Small Objects?
Fujun Han, Junan Chen, Xintong Zhu +4
Multimodal Large Language Models (MLLMs) have shown promising potential in diverse understanding tasks, e.g., image and video analysis, math and physics olympiads. However, they re…
A Unified Study of LoRA Variants: Taxonomy, Review, Codebase, and Empirical Evaluation
Haonan He, Jingqi Ye, Minglei Li +4
Low-Rank Adaptation (LoRA) is a fundamental parameter-efficient fine-tuning method that balances efficiency and performance in large-scale neural networks. However, the proliferati…