11 citations · 17 across the 5 of their papers we have counts for
5 papers
Retrieval Capabilities of Large Language Models Scale with Pretraining FLOPs
Jacob Portes, Connor Jennings, Erica Ji Yuen +2
How does retrieval performance scale with pretraining FLOPs? We benchmark retrieval performance across LLM model sizes from 125 million parameters to 7 billion parameters pretraine…
Sparse Upcycling: Inference Inefficient Finetuning
Sasha Doubov, Nikhil Sardana, Vitaliy Chiley
Small, highly trained, open-source large language models are widely used due to their inference efficiency, but further improving their quality remains a challenge. Sparse upcyclin…
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
Nikhil Sardana, Jacob Portes, Sasha Doubov +1
Large language model (LLM) scaling laws are empirical formulas that estimate changes in model quality as a result of increasing parameter count and training data. However, these fo…
Scalable Neural Data Server: A Data Recommender for Transfer Learning
Tianshi Cao, Sasha Doubov, David Acuna +1
Absence of large-scale labeled data in the practitioner's target domain can be a bottleneck to applying machine learning algorithms in practice. Transfer learning is a popular stra…
Pit30M: A Benchmark for Global Localization in the Age of Self-Driving Cars
Julieta Martinez, Sasha Doubov, Jack Fan +4
We are interested in understanding whether retrieval-based localization approaches are good enough in the context of self-driving vehicles. Towards this goal, we introduce Pit30M,…