5 papers
Toward a First-Principles Update Geometry for the Language-Model Head
Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno +2
Muon motivates designing optimizer geometry around the function of each parameter block and uses the spectral norm for hidden linear layers. For the language-model head, the spectr…
Understanding Adam Requires Better Rotation Dependent Assumptions
Tianyue H. Zhang, Lucas Maes, Alan Milligan +5
Despite its widespread adoption, Adam's advantage over Stochastic Gradient Descent (SGD) lacks a comprehensive theoretical explanation. This paper investigates Adam's sensitivity t…
From Conformal Predictions to Confidence Regions
Charles Guille-Escuret, Eugene Ndiaye
Conformal prediction methodologies have significantly advanced the quantification of uncertainties in predictive models. Yet, the construction of confidence regions for model param…
Finite Sample Confidence Regions for Linear Regression Parameters Using Arbitrary Predictors
Charles Guille-Escuret, Eugene Ndiaye
We explore a novel methodology for constructing confidence regions for parameters of linear models, using predictions from any arbitrary predictor. Our framework requires minimal a…
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection
Charles Guille-Escuret, Pierre-André Noël, Ioannis Mitliagkas +2
Improving the reliability of deployed machine learning systems often involves developing methods to detect out-of-distribution (OOD) inputs. However, existing research often narrow…