805 citations · 1.9k across the 55 of their papers we have counts for
59 papers
Spectral Differential Network Analysis for High-Dimensional Time Series
Michael Hellstern, Byol Kim, Zaid Harchaoui +1
Spectral networks derived from multivariate time series data arise in many domains, from brain science to Earth science. Often, it is of interest to study how these networks change…
Iterated Schrödinger bridge approximation to Wasserstein Gradient Flows
Medha Agarwal, Zaid Harchaoui, Garrett Mulcahy +1
We introduce a novel discretization scheme for Wasserstein gradient flows that involves successively computing Schrödinger bridges with the same marginals. This is different from b…
Distributionally Robust Optimization with Bias and Variance Reduction
Ronak Mehta, Vincent Roulet, Krishna Pillutla +1
We consider the distributionally robust optimization (DRO) problem with spectral risk-based uncertainty set and -divergence penalty. This formulation includes common risk-sensit…
FiLM: Fill-in Language Models for Any-Order Generation
Tianxiao Shen, Hao Peng, Ruoqi Shen +3
Language models have become the backbone of today's AI systems. However, their predominant left-to-right generation limits the use of bidirectional context, which is essential for…
Modified Gauss-Newton Algorithms under Noise
Krishna Pillutla, Vincent Roulet, Sham Kakade +1
Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing. Their nonsmooth counterparts, modified Gauss-Newton or prox-linea…
Faith and Fate: Limits of Transformers on Compositionality
Nouha Dziri, Ximing Lu, Melanie Sclar +13
Transformer large language models (LLMs) have sparked admiration for their exceptional performance on tasks that demand intricate multi-step reasoning. Yet, these models simultaneo…
Confidence Sets under Generalized Self-Concordance
Lang Liu, Zaid Harchaoui
This paper revisits a fundamental problem in statistical inference from a non-asymptotic theoretical viewpoint $\unicode{x2013}$ the construction of confidence sets. We establish a…
Stochastic Optimization for Spectral Risk Measures
Ronak Mehta, Vincent Roulet, Krishna Pillutla +2
Spectral risk objectives - also called -risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and wor…
MAUVE Scores for Generative Models: Theory and Practice
Krishna Pillutla, Lang Liu, John Thickstun +6
Generative artificial intelligence has made significant strides, producing text indistinguishable from human prose and remarkably photorealistic images. Automatically measuring how…
Statistical and Computational Guarantees for Influence Diagnostics
Jillian Fisher, Lang Liu, Krishna Pillutla +2
Influence diagnostics such as influence functions and approximate maximum influence perturbations are popular in machine learning and in AI domain applications. Influence diagnosti…
Iterative Linear Quadratic Optimization for Nonlinear Control: Differentiable Programming Algorithmic Templates
Vincent Roulet, Siddhartha Srinivasa, Maryam Fazel +1
Iterative optimization algorithms depend on access to information about the objective function. In a differentiable programming framework, this information, such as gradients, can…
Orthogonal Statistical Learning with Self-Concordant Loss
Lang Liu, Carlos Cinelli, Zaid Harchaoui
Orthogonal statistical learning and double machine learning have emerged as general frameworks for two-stage statistical prediction in the presence of a nuisance component. We esta…
On Global and Local Convergence of Iterative Linear Quadratic Optimization Algorithms for Discrete Time Nonlinear Control
Vincent Roulet, Siddhartha Srinivasa, Maryam Fazel +1
A classical approach for solving discrete time nonlinear control on a finite horizon consists in repeatedly minimizing linear quadratic approximations of the original problem aroun…
Superquantiles at Work: Machine Learning Applications and Efficient Subgradient Computation
Yassine Laguel, Krishna Pillutla, Jérôme Malick +1
R. Tyrell Rockafellar and collaborators introduced, in a series of works, new regression modeling methods based on the notion of superquantile (or conditional value-at-risk). These…
Superquantile-based learning: a direct approach using gradient-based optimization
Yassine Laguel, Jérôme Malick, Zaid Harchaoui
We consider a formulation of supervised learning that endows models with robustness to distributional shifts from training to testing. The formulation hinges upon the superquantile…
Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates
Nicholas J. Irons, Meyer Scetbon, Soumik Pal +1
Triangular flows, also known as Knöthe-Rosenblatt measure couplings, comprise an important building block of normalizing flow models for generative modeling and density estimation,…
Target Propagation via Regularized Inversion
Vincent Roulet, Zaid Harchaoui
Target Propagation (TP) algorithms compute targets instead of gradients along neural networks and propagate them backward in a way that is similar yet different than gradient back-…
Federated Learning with Superquantile Aggregation for Heterogeneous Data
Krishna Pillutla, Yassine Laguel, Jérôme Malick +1
We present a federated learning framework that is designed to robustly deliver good predictive performance across individual clients with heterogeneous data. The proposed approach…
Score-Based Change Detection for Gradient-Based Learning Machines
Lang Liu, Joseph Salmon, Zaid Harchaoui
The widespread use of machine learning algorithms calls for automatic change detection algorithms to monitor their behavior over time. As a machine learning algorithm learns from a…
Differentiable Programming à la Moreau
Vincent Roulet, Zaid Harchaoui
The notion of a Moreau envelope is central to the analysis of first-order optimization algorithms for machine learning. Yet, it has not been developed and extended to be applied to…
Faster Policy Learning with Continuous-Time Gradients
Samuel Ainsworth, Kendall Lowrey, John Thickstun +2
We study the estimation of policy gradients for continuous-time systems with known dynamics. By reframing policy learning in continuous-time, we show that it is possible construct…
Asymptotics of Discrete Schrödinger Bridges via Chaos Decomposition
Zaid Harchaoui, Lang Liu, Soumik Pal
Consider the problem of matching two independent i.i.d. samples of size from two distributions and in . For an arbitrary continuous cost function, the opt…
First-order Optimization for Superquantile-based Supervised Learning
Yassine Laguel, Jérôme Malick, Zaid Harchaoui
Classical supervised learning via empirical risk (or negative log-likelihood) minimization hinges upon the assumption that the testing distribution coincides with the training dist…
Harmonic Decompositions of Convolutional Networks
Meyer Scetbon, Zaid Harchaoui
We present a description of the function space and the smoothness class associated with a convolutional network using the machinery of reproducing kernel Hilbert spaces. We show th…