37 citations · 54 across the 9 of their papers we have counts for
4 papers · 1 filter
Echo: Simulating Distributed Training At Scale
Yicheng Feng, Yuetao Chen, Kaiwen Chen +7
Simulation offers unique values for both enumeration and extrapolation purposes, and is becoming increasingly important for managing the massive machine learning (ML) clusters and…
Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
Tianwen Wei, Bo Zhu, Liang Zhao +13
In this technical report, we introduce the training methodologies implemented in the development of Skywork-MoE, a high-performance mixture-of-experts (MoE) large language model (L…
LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
Liang Zhao, Tianwen Wei, Liang Zeng +12
We introduce LongSkywork, a long-context Large Language Model (LLM) capable of processing up to 200,000 tokens. We provide a training recipe for efficiently extending context lengt…
Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review
Luoma Ke, Song Tong, Peng Cheng +1
This paper explores the frontiers of large language models (LLMs) in psychology applications. Psychology has undergone several theoretical changes, and the current use of Artificia…