103 citations · 122 across the 18 of their papers we have counts for
4 papers · 1 filter
ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference
Haojie Ouyang, Jianwei Lv, Lei Ren +3
Transformer-based large models excel in natural language processing and computer vision, but face severe computational inefficiencies due to the self-attention's quadratic complexi…
SMAR: Soft Modality-Aware Routing Strategy for MoE-based Multimodal Large Language Models Preserving Language Capabilities
Guoyang Xia, Yifeng Ding, Fengfa Li +4
Mixture of Experts (MoE) architectures have become a key approach for scaling large language models, with growing interest in extending them to multimodal tasks. Existing methods t…
Co-VQA : Answering by Interactive Sub Question Sequence
Ruonan Wang, Yuxi Qian, Fangxiang Feng +2
Most existing approaches to Visual Question Answering (VQA) answer questions directly, however, people usually decompose a complex question into a sequence of simple sub questions…
Multi-stage Pre-training over Simplified Multimodal Pre-training Models
Tongtong Liu, Fangxiang Feng, Xiaojie Wang
Multimodal pre-training models, such as LXMERT, have achieved excellent results in downstream tasks. However, current pre-trained models require large amounts of training data and…