5 citations · 10 across the 15 of their papers we have counts for
4 papers · 1 filter
Don't be lazy: CompleteP enables compute-efficient deep transformers
Nolan Dey, Bin Claire Zhang, Lorenzo Noci +6
We study compute efficiency of LLM training when using different parameterizations, i.e., rules for adjusting model and optimizer hyperparameters (HPs) as model size changes. Some…
Deep Neural Nets as Hamiltonians
Mike Winer, Boris Hanin
Neural networks are complex functions of both their inputs and parameters. Much prior work in deep learning theory analyzes the distribution of network outputs at a fixed a set of…
Optimizing Model Selection for Compound AI Systems
Lingjiao Chen, Jared Quincy Davis, Boris Hanin +4
Compound AI systems that combine multiple LLM calls, such as self-refine and multi-agent-debate, achieve strong performance on many AI tasks. We address a core question in optimizi…
BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
Alan Zhu, Parth Asawa, Jared Quincy Davis +5
As the demand for high-quality data in model training grows, researchers and developers are increasingly generating synthetic data to tune and train LLMs. However, current data gen…