4 papers
Problems with Chinchilla Approach 2: Systematic Biases in IsoFLOP Parabola Fits
Eric Czech, Zhiwei Xu, Yael Elmatad +2
Chinchilla Approach 2 is among the most widely used methods for fitting neural scaling laws. Its parabolic approximation introduces systematic biases in compute-optimal allocation…
Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimization
Weichi Yao, Bianca Dumitrascu, Bryan R. Goldsmith +1
Active data acquisition is central to many learning and optimization tasks in deep neural networks, yet remains challenging because most approaches rely on predictive uncertainty e…
An Efficient Smoothing Damped Newton Method for Large-Scale Mathematical Programs with Equilibrium Constraints
Yixin Wang, Qingna Li, Liwei Zhang
Bilevel hyperparameter optimization has received growing attention thanks to the fast development of machine learning. Due to the tremendous size of data sets, the scale of bilevel…
Faster randomized partial trace estimation
Tyler Chen, Robert Chen, Kevin Li +3
We develop randomized matrix-free algorithms for estimating partial traces, a generalization of the trace arising in quantum physics and chemistry. Our algorithm improves on the ty…