11 papers
Adjusted Scores for Discrete Langevin Algorithms
Armand Gissler, Saeed Saremi, Francis Bach
Sampling from discrete distributions is a ubiquitous task in machine learning, recently revisited by the emergence of discrete diffusion models. While Langevin algorithms constitut…
Two-stage stochastic algorithm for solving large-scale (non)-convex separable optimization problems under affine constraints
Benjamin Dubois-Taine, Laurent Pfeiffer, Nadia Oudjane +2
We consider nonsmooth optimization problems under affine constraints, where the objective consists of the average of the component functions of a large number of agents, and we…
A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage
Francis Bach
We consider supervised learning problems in which set predictions provide explicit uncertainty estimates. Using Choquet integrals (a.k.a. Lov{á}sz extensions), we propose a convex…
On the Effectiveness of the z-Transform Method in Quadratic Optimization
Francis Bach
The z-transform of a sequence is a classical tool used within signal processing, control theory, computer science, and electrical engineering. It allows for studying sequences from…
Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law
Frederik Kunstner, Francis Bach
Recent works have highlighted optimization difficulties faced by gradient descent in training the first and last layers of transformer-based language models, which are overcome by…
Spectral structure learning for clinical time series
Ivan Lerner, Anita Burgun, Francis Bach
We develop and evaluate a structure learning algorithm for clinical time series. Clinical time series are multivariate time series observed in multiple patients and irregularly sam…
An Uncertainty Principle for Linear Recurrent Neural Networks
Alexandre François, Antonio Orvieto, Francis Bach
We consider linear recurrent neural networks, which have become a key building block of sequence modeling due to their ability for stable and effective long-range modeling. In this…
Convergence of Shallow ReLU Networks on Weakly Interacting Data
Léo Dana, Francis Bach, Loucas Pillaud-Vivien
We analyse the convergence of one-hidden-layer ReLU networks trained by gradient flow on data points. Our main contribution leverages the high dimensionality of the ambient spa…
Optimizing Estimators of Squared Calibration Errors in Classification
Sebastian G. Gruber, Francis Bach
In this work, we propose a mean-squared error-based risk that enables the comparison and optimization of estimators of squared calibration errors in practical settings. Improving t…
Enhanced Feature Learning via Regularisation: Integrating Neural Networks and Kernel Methods
Bertille Follain, Francis Bach
We propose a new method for feature learning and function estimation in supervised learning via regularised empirical risk minimisation. Our approach considers functions as expecta…
Screening for a Reweighted Penalized Conditional Gradient Method
Yifan Sun, Francis Bach
The conditional gradient method (CGM) is widely used in large-scale sparse convex optimization, having a low per iteration computational cost for structured sparse regularizers and…