7 papers
OTel: Open Telco AI Datasets, Benchmarks, and Models
Farbod Tavakkoli, Gregory Diamos, Kenneth Church +19
We present Open Telco (OTel), an open telecom AI resource that releases derived telecom datasets for retrieval, reranking, instruction tuning, and safety/abstention, together with…
Essential-Web v1.0: 24T tokens of organized web data
Essential AI, :, Andrew Hojel +22
Data plays the most prominent role in how language models acquire skills and knowledge. The lack of massive, well-organized pre-training datasets results in costly and inaccessible…
Practical Efficiency of Muon for Pretraining
Essential AI, :, Ishaan Shah +22
We demonstrate that Muon, the simplest instantiation of a second-order optimizer, explicitly expands the Pareto frontier over AdamW on the compute-time tradeoff. We find that Muon…
Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models
Yanbin Yin, Kun Zhou, Zhen Wang +11
The recent explosion of large language models (LLMs), each with its own general or specialized strengths, makes scalable, reliable benchmarking more urgent than ever. Standard prac…
Rethinking Reflection in Pre-Training
Essential AI, :, Darsh J Shah +26
A language model's ability to reflect on its own reasoning provides a key advantage for solving complex problems. While most recent research has focused on how this ability develop…
Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models
Somanshu Singla, Zhen Wang, Tianyang Liu +3
Aligning Large Language Models (LLMs) traditionally relies on costly training and human preference annotations. Self-alignment seeks to reduce these expenses by enabling models to…
Unsupervised Contrastive Learning of Image Representations from Ultrasound Videos with Hard Negative Mining
Soumen Basu, Somanshu Singla, Mayank Gupta +3
Rich temporal information and variations in viewpoints make video data an attractive choice for learning image representations using unsupervised contrastive learning (UCL) techniq…