5 papers
Reassessing How to Compare and Improve the Calibration of Machine Learning Models
Muthu Chidambaram, Rong Ge
A machine learning model is calibrated if its predicted probability for an outcome matches the observed frequency for that outcome conditional on the model prediction. This propert…
Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensing
Shuyao Li, Yu Cheng, Ilias Diakonikolas +3
Finding an approximate second-order stationary point (SOSP) is a well-studied and fundamental problem in stochastic nonconvex optimization with many applications in machine learnin…
Linear Transformers are Versatile In-Context Learners
Max Vladymyrov, Johannes von Oswald, Mark Sandler +1
Recent research has demonstrated that transformers, particularly linear attention models, implicitly execute gradient-descent-like algorithms on data provided in-context during the…
Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input
Ziang Chen, Rong Ge
In this work, we study the mean-field flow for learning subspace-sparse polynomials using stochastic gradient descent and two-layer neural networks, where the input distribution is…
For Better or For Worse? Learning Minimum Variance Features With Label Augmentation
Muthu Chidambaram, Rong Ge
Data augmentation has been pivotal in successfully training deep learning models on classification tasks over the past decade. An important subclass of data augmentation techniques…