1.3k citations · 3.8k across the 222 of their papers we have counts for
19 papers · 1 filter
Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods
Kimon Antonakopoulos, Ali Kavis, Volkan Cevher
This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions. Our algorithm achieves convergence when the o…
Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization
Ali Kavis, Stratis Skoulakis, Kimon Antonakopoulos +2
We propose an adaptive variance-reduction method, called AdaSpider, for minimization of -smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines…
Sound and Complete Verification of Polynomial Networks
Elias Abad Rocamora, Mehmet Fatih Sahin, Fanghui Liu +2
Polynomial Networks (PNs) have demonstrated promising performance on face and image recognition recently. However, robustness of PNs is unclear and thus obtaining certificates beco…
Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study
Yongtao Wu, Zhenyu Zhu, Fanghui Liu +2
Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds. The study on NTK has been devoted to typical neural…
Understanding Deep Neural Function Approximation in Reinforcement Learning via -Greedy Exploration
Fanghui Liu, Luca Viano, Volkan Cevher
This paper provides a theoretical study of deep neural function approximation in reinforcement learning (RL) with the -greedy exploration under the online setting. This problem…
Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning
Paul Rolland, Luca Viano, Norman Schuerhoff +2
While Reinforcement Learning (RL) aims to train an agent from a reward function in a given environment, Inverse Reinforcement Learning (IRL) seeks to recover the reward function fr…
DiGress: Discrete Denoising diffusion for graph generation
Clement Vignac, Igor Krawczuk, Antoine Siraudin +3
This work introduces DiGress, a discrete denoising diffusion model for generating graphs with categorical node and edge attributes. Our model utilizes a discrete diffusion process…
Proximal Point Imitation Learning
Luca Viano, Angeliki Kamoutsi, Gergely Neu +2
This work develops new algorithms with rigorous efficiency guarantees for infinite horizon imitation learning (IL) with linear function approximation without restrictive coherence…
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)
Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos +1
We study the average robustness notion in deep neural networks in (selected) wide and narrow, deep and shallow, as well as lazy and non-lazy training settings. We prove that in the…
Generalization Properties of NAS under Activation and Skip Connection Search
Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos +1
Neural Architecture Search (NAS) has fostered the automatic discovery of state-of-the-art neural architectures. Despite the progress achieved with NAS, so far there is little atten…
Adversarial Audio Synthesis with Complex-valued Polynomial Networks
Yongtao Wu, Grigorios G Chrysos, Volkan Cevher
Time-frequency (TF) representations in audio synthesis have been increasingly modeled with real-valued networks. However, overlooking the complex-valued nature of TF representation…
No-Regret Learning in Games with Noisy Feedback: Faster Rates and Adaptivity via Learning Rate Separation
Yu-Guan Hsieh, Kimon Antonakopoulos, Volkan Cevher +1
We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algori…
A unified stochastic approximation framework for learning in games
Panayotis Mertikopoulos, Ya-Ping Hsieh, Volkan Cevher
We develop a flexible stochastic approximation framework for analyzing the long-run behavior of learning in games (both continuous and finite). The proposed analysis template incor…
High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize
Ali Kavis, Kfir Yehuda Levy, Volkan Cevher
In this paper, we propose a new, simplified high probability analysis of AdaGrad for smooth, non-convex problems. More specifically, we focus on a particular accelerated gradient (…
Score matching enables causal discovery of nonlinear additive noise models
Paul Rolland, Volkan Cevher, Matthäus Kleindessner +4
This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models. Using score matching algorithms as a bu…
The Spectral Bias of Polynomial Neural Networks
Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos +2
Polynomial neural networks (PNNs) have been recently shown to be particularly effective at image generation and face recognition, where high-frequency information is critical. Prev…
Controlling the Complexity and Lipschitz Constant improves polynomial nets
Zhenyu Zhu, Fabian Latorre, Grigorios G Chrysos +1
While the class of Polynomial Nets demonstrates comparable performance to neural networks (NN), it currently has neither theoretical generalization characterization nor robustness…
Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization
Gideon Dresdner, Maria-Luiza Vladarean, Gunnar Rätsch +3
We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this…
On the Complexity of a Practical Primal-Dual Coordinate Method
Ahmet Alacaoglu, Volkan Cevher, Stephen J. Wright
We prove complexity bounds for the primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD), which has been shown to obtain good practical performance for s…