5 papers · 1 filter
MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models
Jie Cao, Tianwei Lin, Bo Yuan +7
Recent studies integrate Low-Rank Adaptation (LoRA) and Mixture-of-Experts (MoE) to further enhance the performance of parameter-efficient fine-tuning (PEFT) methods in Large Langu…
Fast Thinking for Large Language Models
Haoyu Zheng, Zhuonan Wang, Yuqian Yuan +7
Reasoning-oriented Large Language Models (LLMs) often rely on generating explicit tokens step by step, and their effectiveness typically hinges on large-scale supervised fine-tunin…
Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts
Xiangnan Chen, Yuancheng Fang, Qian Xiao +5
Multimodal Large Language Models (MLLMs) have garnered significant attention for their strong visual-semantic understanding. Most existing chart benchmarks evaluate MLLMs' ability…
TeamLoRA: Boosting Low-Rank Adaptation with Expert Collaboration and Competition
Tianwei Lin, Jiang Liu, Wenqiao Zhang +9
While Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have effectively addressed GPU memory constraints during fine-tuning, their performance often falls short, especially…
I3: Intent-Introspective Retrieval Conditioned on Instructions
Kaihang Pan, Juncheng Li, Wenjie Wang +7
Recent studies indicate that dense retrieval models struggle to perform well on a wide variety of retrieval tasks that lack dedicated training data, as different retrieval tasks of…