1.3k citations · 3.8k across the 222 of their papers we have counts for
22 papers · 1 filter
High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization
Yihang Chen, Fanghui Liu, Taiji Suzuki +1
This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of h…
Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks
Fanghui Liu, Leello Dadi, Volkan Cevher
Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks as the curse of dimensionality (CoD) cannot be evad…
Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks
Jiayuan Ye, Zhenyu Zhu, Fanghui Liu +2
We analytically investigate how over-parameterization of models in randomized machine learning algorithms impacts the information leakage about their training data. Specifically, w…
What can online reinforcement learning with function approximation benefit from general coverage conditions?
Fanghui Liu, Luca Viano, Volkan Cevher
In online reinforcement learning (RL), instead of employing standard structural assumptions on Markov decision processes (MDPs), using a certain coverage condition (original from o…
On the Double Descent of Random Features Models Trained with SGD
Fanghui Liu, Johan A. K. Suykens, Volkan Cevher
We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime. In this…
Convergence of adaptive algorithms for weakly convex constrained optimization
Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective. We prove the $\mathcal{\tilde O}(t^…
A new regret analysis for Adam-type algorithms
Ahmet Alacaoglu, Yura Malitsky, Panayotis Mertikopoulos +1
In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.). In practice, these algorithms are used with a constant first-order moment parame…
Kernel Conjugate Gradient Methods with Random Projections
Junhong Lin, Volkan Cevher
We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random p…
Adversarially Robust Optimization with Gaussian Processes
Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka +1
In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we requi…
Optimal Rates of Sketched-regularized Algorithms for Least-Squares Regression over Hilbert Spaces
Junhong Lin, Volkan Cevher
We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kern…
Robust Maximization of Non-Submodular Objectives
Ilija Bogunovic, Junyao Zhao, Volkan Cevher
We study the problem of maximizing a monotone set function subject to a cardinality constraint in the setting where some number of elements is deleted from the returned set…
Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms
Junhong Lin, Volkan Cevher
We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distrib…
Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces
Junhong Lin, Alessandro Rudi, Lorenzo Rosasco +1
In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space. We inve…
Phase Transitions in the Pooled Data Problem
Jonathan Scarlett, Volkan Cevher
In this paper, we study the pooled data problem of identifying the labels associated with a large collection of items, based on a sequence of pooled tests revealing the counts of e…
Robust Submodular Maximization: A Non-Uniform Partitioning Approach
Ilija Bogunovic, Slobodan Mitrović, Jonathan Scarlett +1
We study the problem of maximizing a monotone submodular function subject to a cardinality constraint , with the added twist that a number of items from the returned set may…
Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimization
Jonathan Scarlett, Ilijia Bogunovic, Volkan Cevher
In this paper, we consider the problem of sequentially optimizing a black-box function based on noisy samples and bandit feedback. We assume that is smooth in the sense of…
Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimation
Ilija Bogunovic, Jonathan Scarlett, Andreas Krause +1
We present a new algorithm, truncated variance reduction (TruVaR), that treats Bayesian optimization (BO) and level-set estimation (LSE) with Gaussian processes in a unified fashio…
Time-Varying Gaussian Process Bandit Optimization
Ilija Bogunovic, Jonathan Scarlett, Volkan Cevher
We consider the sequential Bayesian optimization problem with bandit feedback, adopting a formulation that allows for the reward function to vary with time. We model the reward fun…
Scalable sparse covariance estimation via self-concordance
Anastasios Kyrillidis, Rabeeh Karimi Mahabadi, Quoc Tran-Dinh +1
We consider the class of convex minimization problems, composed of a self-concordant function, such as the metric, a convex data fidelity term and, a regulari…
Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods
Hemant Tyagi, Volkan Cevher
We consider the problem of learning multi-ridge functions of the form f(x) = g(Ax) from point evaluations of f. We assume that the function f is defined on an l_2-ball in R^d, g is…
Composite Self-Concordant Minimization
Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher
We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal op…
A proximal Newton framework for composite minimization: Graph learning without Cholesky decompositions and matrix inversions
Quoc Tran Dinh, Anastasios Kyrillidis, Volkan Cevher
We propose an algorithmic framework for convex minimization problems of a composite function with two terms: a self-concordant function and a possibly nonsmooth regularization term…