7 papers
Can an Individual Manipulate the Collective Decisions of Multi-Agents?
Fengyuan Liu, Rui Zhao, Shuo Chen +4
Individual Large Language Models (LLMs) have demonstrated significant capabilities across various domains, such as healthcare and law. Recent studies also show that coordinated mul…
True Multimodal In-Context Learning Needs Attention to the Visual Context
Shuo Chen, Jianzhe Liu, Zhen Han +5
Multimodal Large Language Models (MLLMs), built on powerful language backbones, have enabled Multimodal In-Context Learning (MICL)-adapting to new tasks from a few multimodal demon…
Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
Yilun Liu, Yunpu Ma, Yuetian Lu +3
Mixture-of-Experts (MoE) benefits from a dynamic routing mechanism among their specialized experts, which existing Parameter- Efficient Fine-Tuning (PEFT) strategies fail to levera…
METok: Multi-Stage Event-based Token Compression for Efficient Long Video Understanding
Mengyue Wang, Shuo Chen, Kristian Kersting +2
Recent advances in Video Large Language Models (VLLMs) have significantly enhanced their ability to understand video content. Nonetheless, processing long videos remains challengin…
Supposedly Equivalent Facts That Aren't? Entity Frequency in Pre-training Induces Asymmetry in LLMs
Yuan He, Bailan He, Zifeng Ding +8
Understanding and mitigating hallucinations in Large Language Models (LLMs) is crucial for ensuring reliable content generation. While previous research has primarily focused on "w…
PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
Yilun Liu, Yunpu Ma, Shuo Chen +4
The Mixture-of-Experts (MoE) paradigm has emerged as a powerful approach for scaling transformers with improved resource utilization. However, efficiently fine-tuning MoE models re…