4 papers
The Finetuner's Fallacy: When to Pretrain with Your Finetuning Data
Christina Baek, Ricardo Pio Monti, David Schwab +31
Real-world model deployments demand strong performance on narrow domains where data is often scarce. Typically, practitioners finetune models to specialize them, but this risks ove…
ÜberWeb: Insights from Multilingual Curation for a 20-Trillion-Token Dataset
DatologyAI, :, Aldo Gael Carranza +32
Multilinguality is a core capability for modern foundation models, yet training high-quality multilingual models remains challenging due to uneven data availability across language…
DatBench: Discriminative, Faithful, and Efficient VLM Evaluations
DatologyAI, :, Siddharth Joshi +30
Empirical evaluation serves as the primary compass guiding research progress in foundation models. Despite a large body of work focused on training frontier vision-language models…
Luxical: High-Speed Lexical-Dense Text Embeddings
DatologyAI, :, Luke Merrick +31
Frontier language model quality increasingly hinges on our ability to organize web-scale text corpora for training. Today's dominant tools trade off speed and flexibility: lexical…