9 papers
An Empirical Study of Parameter-Efficient Fine-Tuning for Mixture-of-Experts LLMs
Minglei Li, Peng Ye, Haonan He +3
Parameter-efficient fine-tuning (PEFT) methods are critical for adapting large models under resource-constrained settings. However, as mixture-of-experts (MoE) architectures, which…
MoE-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation
Qingyu Yang, Haonan He, Minglei Li +4
Mixture-of-Experts (MoE) architectures have been widely adopted in large language models, yet parameter-efficient fine-tuning (PEFT) for MoE models remains underexplored. Existing…
DiBS: Diffusion-Informed Branch Selection
Bo Liu, Yuan Xie, Yuan Gao +4
Sudoku is a representative constraint satisfaction problem that requires global structural reasoning under strict discrete constraints. The existing works of solving Sudoku mainly…
Can Multimodal Large Language Models Truly Understand Small Objects?
Fujun Han, Junan Chen, Xintong Zhu +4
Multimodal Large Language Models (MLLMs) have shown promising potential in diverse understanding tasks, e.g., image and video analysis, math and physics olympiads. However, they re…
Learning from Execution: Self-Evolving Memory for Private-Library Code Generation
Mofei Li, Taozhi Chen, Guowei Yang +1
Large Language Models (LLMs) have achieved strong performance on general code generation, but their effectiveness drops sharply in enterprise settings where software development re…
A Unified Study of LoRA Variants: Taxonomy, Review, Codebase, and Empirical Evaluation
Haonan He, Jingqi Ye, Minglei Li +4
Low-Rank Adaptation (LoRA) is a fundamental parameter-efficient fine-tuning method that balances efficiency and performance in large-scale neural networks. However, the proliferati…
FRISM: Fine-Grained Reasoning Injection via Subspace-Level Model Merging for Vision-Language Models
Chenyu Huang, Peng Ye, Xudong Tan +4
Efficiently enhancing the reasoning capabilities of Vision-Language Models (VLMs) by merging them with Large Reasoning Models (LRMs) has emerged as a promising direction. However,…
SCI-Verifier: Scientific Verifier with Thinking
Shenghe Zheng, Chenyu Huang, Fangchen Yu +8
As large language models (LLMs) are increasingly applied to scientific reasoning, the complexity of answer formats and the diversity of equivalent expressions make answer verificat…
HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?
Fangchen Yu, Haiyuan Wan, Qianjia Cheng +14
Recently, the physical capabilities of (M)LLMs have garnered increasing attention. However, existing benchmarks for physics suffer from two major gaps: they neither provide systema…