40 citations · 50 across the 13 of their papers we have counts for
5 papers · 1 filter
Unlocking the Potential of Model Merging for Low-Resource Languages
Mingxu Tao, Chen Zhang, Quzhe Huang +4
Adapting large language models (LLMs) to new languages typically involves continual pre-training (CT) followed by supervised fine-tuning (SFT). However, this CT-then-SFT approach s…
Can Perplexity Reflect Large Language Model's Ability in Long Text Understanding?
Yutong Hu, Quzhe Huang, Mingxu Tao +2
Recent studies have shown that Large Language Models (LLMs) have the potential to process extremely long text. Many works only evaluate LLMs' long-text processing ability on the la…
Harder Tasks Need More Experts: Dynamic Routing in MoE Models
Quzhe Huang, Zhenwei An, Nan Zhuang +7
In this paper, we introduce a novel dynamic expert selection framework for Mixture of Experts (MoE) models, aiming to enhance computational efficiency and model performance by adju…
Probing Multimodal Large Language Models for Global and Local Semantic Representations
Mingxu Tao, Quzhe Huang, Kun Xu +3
The advancement of Multimodal Large Language Models (MLLMs) has greatly accelerated the development of applications in understanding integrated texts and images. Recent works lever…
Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering
Mingxu Tao, Dongyan Zhao, Yansong Feng
Open-ended question answering requires models to find appropriate evidence to form wellreasoned, comprehensive and helpful answers. In practical applications, models also need to e…