6 citations · 8 across the 11 of their papers we have counts for
6 papers · 1 filter
Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need
Bo-Wen Zhang, Yan Yan, Boxiang Yang +2
While scaling laws optimize training configurations for large language models (LLMs) through experiments on smaller or early-stage models, they fail to predict emergent abilities d…
CCI3.0-HQ: a large-scale Chinese dataset of high quality designed for pre-training large language models
Liangdong Wang, Bo-Wen Zhang, Chengwei Wu +7
We present CCI3.0-HQ (https://huggingface.co/datasets/BAAI/CCI3-HQ), a high-quality 500GB subset of the Chinese Corpora Internet 3.0 (CCI3.0)(https://huggingface.co/datasets/BAAI/C…
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data
Shuhao Gu, Jialing Zhang, Siyuan Zhou +23
Recently, Vision-Language Models (VLMs) have achieved remarkable progress in multimodal tasks, and multimodal instruction data serves as the foundation for enhancing VLM capabiliti…
Aquila2 Technical Report
Bo-Wen Zhang, Liangdong Wang, Jijie Li +6
This paper introduces the Aquila2 series, which comprises a wide range of bilingual models with parameter sizes of 7, 34, and 70 billion. These models are trained based on an innov…
AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
Bo-Wen Zhang, Liangdong Wang, Ye Yuan +24
In recent years, with the rapid application of large language models across various fields, the scale of these models has gradually increased, and the resources required for their…
InfinityMATH: A Scalable Instruction Tuning Dataset in Programmatic Mathematical Reasoning
Bo-Wen Zhang, Yan Yan, Lin Li +1
Recent advancements in Chain-of-Thoughts (CoT) and Program-of-Thoughts (PoT) methods have greatly enhanced language models' mathematical reasoning capabilities, facilitating their…