8 citations · 16 across the 8 of their papers we have counts for
8 papers
Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
Alon Albalak, Duy Phung, Nathan Lile +8
Increasing interest in reasoning models has led math to become a prominent testing ground for algorithmic and methodological improvements. However, existing open math datasets eith…
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
Violet Xiang, Charlie Snell, Kanishk Gandhi +11
We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive…
Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models
Alex Havrilla, Andrew Dai, Laura O'Mahony +17
Synthetic data generation with Large Language Models is a promising paradigm for augmenting natural data over a nearly infinite range of tasks. Given this variety, direct compariso…
Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic
Zaid Alyafeai, Michael Pieler, Hannah Teufel +8
Large Language Models (LLMs) have shown impressive results in multiple domains of natural language processing (NLP) but are mainly focused on the English language. Recently, more L…
Rephrasing natural text data with different languages and quality levels for Large Language Model pre-training
Michael Pieler, Marco Bellagente, Hannah Teufel +9
Recently published work on rephrasing natural text data for pre-training LLMs has shown promising results when combining the original dataset with the synthetically rephrased data.…
Generative Reward Models
Dakota Mahan, Duy Van Phung, Rafael Rafailov +6
Reinforcement Learning from Human Feedback (RLHF) has greatly improved the performance of modern Large Language Models (LLMs). The RLHF process is resource-intensive and technicall…
Stable Code Technical Report
Nikhil Pinnaparaju, Reshinth Adithyan, Duy Phung +8
We introduce Stable Code, the first in our new-generation of code language models series, which serves as a general-purpose base code language model targeting code completion, reas…
Stable LM 2 1.6B Technical Report
Marco Bellagente, Jonathan Tow, Dakota Mahan +16
We introduce StableLM 2 1.6B, the first in a new generation of our language model series. In this technical report, we present in detail the data and training procedure leading to…