6 papers
Turnslide: Scalable Multi-Turn Data Synthesis by Walking a Finite-State Machine
Aaron Fainman, Gabriela Kadlecová, Maciej Gryka +7
Small language models are inexpensive to serve and can run on private infrastructure, but base models are often not good enough at multi-turn tool calling, and fine-tuning them nee…
MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages
Maximilian Idahl, Jörg Tiedemann, Sampo Pyysalo +19
Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic parallel corpus with approxi…
An Open-Source Training Dataset for Foundation Models for Black-box Optimization
Aaron Klein, Herilalaina Rakotoarison, Luca Thale-Bombien +1
Most black-box optimization methods require extensive hyperparameter tuning, often limiting their ability to generalize across different optimization domains. Foundation models for…
Impermanent: A Live Benchmark for Temporal Generalization in Time Series Forecasting
Azul Garza, Renée Rosillo, Rodrigo Mendoza-Smith +5
Recent advances in time-series forecasting increasingly rely on pre-trained foundation-style models. While these models often claim broad generalization, existing evaluation protoc…
MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources
Huu Nguyen, Victor May, Harsh Raj +14
We present MixtureVitae, an open-access pretraining corpus built to minimize legal risk while providing strong downstream performance. MixtureVitae follows a permissive-first, risk…
Open-sci-ref-0.02: open reference baselines for reproducible language model and dataset comparison
Marianna Nezhurina, Harsh Raj, Jörg Franke +7
Reliable progress in language modeling depends on knowing whether a new dataset, architecture, or training recipe is genuinely better than existing alternatives. To foster reproduc…