11 citations · 15 across the 7 of their papers we have counts for
7 papers
Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs
Liang Zeng, Yongcong Li, Yuzhen Xiao +8
Software engineering (SWE) has recently emerged as a crucial testbed for next-generation LLM agents, demanding inherent capabilities in two critical dimensions: sustained iterative…
Skywork Open Reasoner 1 Technical Report
Jujie He, Jiacai Liu, Chris Yuhao Liu +14
The success of DeepSeek-R1 underscores the significant role of reinforcement learning (RL) in enhancing the reasoning capabilities of large language models (LLMs). In this work, we…
Optimization Hyper-parameter Laws for Large Language Models
Xingyu Xie, Kuangyu Ding, Shuicheng Yan +2
Large Language Models have driven significant AI advancements, yet their training is resource-intensive and highly sensitive to hyper-parameter selection. While scaling laws provid…
Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
Liang Zeng, Liangjun Zhong, Liang Zhao +9
In this paper, we investigate the underlying factors that potentially enhance the mathematical reasoning capabilities of large language models (LLMs). We argue that the data scalin…
Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
Tianwen Wei, Bo Zhu, Liang Zhao +13
In this technical report, we introduce the training methodologies implemented in the development of Skywork-MoE, a high-performance mixture-of-experts (MoE) large language model (L…
LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
Liang Zhao, Tianwen Wei, Liang Zeng +12
We introduce LongSkywork, a long-context Large Language Model (LLM) capable of processing up to 200,000 tokens. We provide a training recipe for efficiently extending context lengt…
Skywork: A More Open Bilingual Foundation Model
Tianwen Wei, Liang Zhao, Lichang Zhang +27
In this technical report, we present Skywork-13B, a family of large language models (LLMs) trained on a corpus of over 3.2 trillion tokens drawn from both English and Chinese texts…