2 citations · 5 across the 43 of their papers we have counts for
6 papers · 1 filter
LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
Xiaoye Qu, Daize Dong, Xuyang Hu +3
Recently, inspired by the concept of sparsity, Mixture-of-Experts (MoE) models have gained increasing popularity for scaling model size while keeping the number of activated parame…
CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling
Jihai Zhang, Xiaoye Qu, Tong Zhu +1
Contrastive Language-Image Pre-training (CLIP) has become a cornerstone in multimodal intelligence. However, recent studies discovered that CLIP can only encode one aspect of the f…
ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM
Zhaochen Su, Jun Zhang, Xiaoye Qu +6
Large language models (LLMs) have achieved impressive advancements across numerous disciplines, yet the critical issue of knowledge conflicts, a major source of hallucinations, has…
Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts
Tong Zhu, Daize Dong, Xiaoye Qu +3
Mixture-of-Experts (MoE) models have shown remarkable capability in instruction tuning, especially when the number of tasks scales. However, previous methods simply merge all train…
Timo: Towards Better Temporal Reasoning for Language Models
Zhaochen Su, Jun Zhang, Tong Zhu +4
Reasoning about time is essential for Large Language Models (LLMs) to understand the world. Previous works focus on solving specific tasks, primarily on time-sensitive question ans…
A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching
Dong Yao
Sentence semantic matching is a research hotspot in natural language processing, which is considerably significant in various key scenarios, such as community question answering, s…