2 citations · 3 across the 11 of their papers we have counts for
7 papers · 1 filter
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
Mingyang Song, Zhaochen Su, Xiaoye Qu +2
Process-level Reward Models (PRMs) are crucial for complex reasoning and decision-making tasks, where each intermediate step plays an important role in the reasoning process. Since…
LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
Xiaoye Qu, Daize Dong, Xuyang Hu +3
Recently, inspired by the concept of sparsity, Mixture-of-Experts (MoE) models have gained increasing popularity for scaling model size while keeping the number of activated parame…
ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM
Zhaochen Su, Jun Zhang, Xiaoye Qu +6
Large language models (LLMs) have achieved impressive advancements across numerous disciplines, yet the critical issue of knowledge conflicts, a major source of hallucinations, has…
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
Tong Zhu, Xiaoye Qu, Daize Dong +4
Mixture-of-Experts (MoE) has gained increasing popularity as a promising framework for scaling up large language models (LLMs). However, training MoE from scratch in a large-scale…
Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts
Tong Zhu, Daize Dong, Xiaoye Qu +3
Mixture-of-Experts (MoE) models have shown remarkable capability in instruction tuning, especially when the number of tasks scales. However, previous methods simply merge all train…
Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?
Zhaochen Su, Juntao Li, Jun Zhang +6
Temporal reasoning is fundamental for large language models (LLMs) to comprehend the world. Current temporal reasoning datasets are limited to questions about single or isolated ev…