2k citations · 6.6k across the 78 of their papers we have counts for
13 papers · 1 filter
Self-Attention in Colors: Another Take on Encoding Graph Structure in Transformers
Romain Menegaux, Emmanuel Jehanno, Margot Selosse +1
We introduce a novel self-attention mechanism, which we call CSA (Chromatic Self-Attention), which extends the notion of attention scores to attention _filters_, independently modu…
Sequential Counterfactual Risk Minimization
Houssam Zenati, Eustache Diemert, Matthieu Martin +2
Counterfactual Risk Minimization (CRM) is a framework for dealing with the logged bandit feedback problem, where the goal is to improve a logging policy using offline data. In this…
The Spectral Bias of Polynomial Neural Networks
Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos +2
Polynomial neural networks (PNNs) have been recently shown to be particularly effective at image generation and face recognition, where high-frequency information is critical. Prev…
Efficient Kernel UCB for Contextual Bandits
Houssam Zenati, Alberto Bietti, Eustache Diemert +3
In this paper, we tackle the computational efficiency of kernelized UCB algorithms in contextual bandits. While standard methods require a O(CT^3) complexity where T is the horizon…
Residual Reinforcement Learning from Demonstrations
Minttu Alakuijala, Gabriel Dulac-Arnold, Julien Mairal +2
Residual reinforcement learning (RL) has been proposed as a way to solve challenging robotic tasks by adapting control actions from a conventional feedback controller to maximize a…
GraphiT: Encoding Graph Structure in Transformers
Grégoire Mialon, Dexiong Chen, Margot Selosse +1
We show that viewing graphs as sets of node features and incorporating structural and positional information into a transformer architecture is able to outperform representations l…
Beyond Tikhonov: Faster Learning with Self-Concordant Losses via Iterative Regularization
Gaspard Beugnot, Julien Mairal, Alessandro Rudi
The theory of spectral filtering is a remarkable tool to understand the statistical properties of learning with kernels. For least squares, it allows to derive various regularizati…
A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention
Grégoire Mialon, Dexiong Chen, Alexandre d'Aspremont +1
We address the problem of learning on sets of features, motivated by the need of performing pooling operations in long biological sequences of varying sizes, with long-range depend…
Screening Data Points in Empirical Risk Minimization via Ellipsoidal Regions and Safe Loss Functions
Grégoire Mialon, Alexandre d'Aspremont, Julien Mairal
We design simple screening tests to automatically discard data samples in empirical risk minimization without losing optimization guarantees. We derive loss functions that produce…
Structured sparsity through convex optimization
Francis Bach, Rodolphe Jenatton, Julien Mairal +1
Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. While naturally cast as a combinatorial optimization problem, variable or…
Optimization with Sparsity-Inducing Penalties
Francis Bach, Rodolphe Jenatton, Julien Mairal +1
Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. They were first dedicated to linear variable selection but numerous extens…
Network Flow Algorithms for Structured Sparsity
Julien Mairal, Rodolphe Jenatton, Guillaume Obozinski +1
We consider a class of learning problems that involve a structured sparsity-inducing norm defined as the sum of -norms over groups of variables. Whereas a lot of effor…
Convex Sparse Matrix Factorizations
Francis Bach, Julien Mairal, Jean Ponce
We present a convex formulation of dictionary learning for sparse signal decomposition. Convexity is obtained by replacing the usual explicit upper bound on the dictionary size by…