70 citations · 445 across the 17 of their papers we have counts for
19 papers
GPU Accelerated Sub-Sampled Newton's Method
Sudhir B. Kylasa, Farbod Roosta-Khorasani, Michael W. Mahoney +1
First order methods, which solely rely on gradient information, are commonly used in diverse machine learning (ML) and data analysis (DA) applications. This is attributed to the si…
Inexact Non-Convex Newton-Type Methods
Zhewei Yao, Peng Xu, Farbod Roosta-Khorasani +1
For solving large-scale non-convex problems, we propose inexact variants of trust region and adaptive cubic regularization methods, which, to increase efficiency, incorporate vario…
Out-of-sample extension of graph adjacency spectral embedding
Keith Levin, Farbod Roosta-Khorasani, Michael W. Mahoney +1
Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to apply a learned embedding to observations not seen in the initial trai…
Optimization Methods for Inverse Problems
Nan Ye, Farbod Roosta-Khorasani, Tiangang Cui
Optimization plays an important role in solving many inverse problems. Indeed, the task of inversion often either involves or is fully cast as a solution of an optimization problem…
Invariance of Weight Distributions in Rectified MLPs
Russell Tsuchida, Farbod Roosta-Khorasani, Marcus Gallagher
An interesting approach to analyzing neural networks that has received renewed attention is to examine the equivalent kernel of the neural network. This is based on the fact that a…
GIANT: Globally Improved Approximate Newton Method for Distributed Optimization
Shusen Wang, Farbod Roosta-Khorasani, Peng Xu +1
For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method. At…
Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study
Peng Xu, Farbod Roosta-Khorasani, Michael W. Mahoney
While first-order optimization methods such as stochastic gradient descent (SGD) are popular in machine learning (ML), they come with well-known deficiencies, including relatively-…
Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information
Peng Xu, Fred Roosta, Michael W. Mahoney
We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact…
Union of Intersections (UoI) for Interpretable Data Driven Discovery and Prediction
Kristofer E. Bouchard, Alejandro F. Bujan, Farbod Roosta-Khorasani +7
The increasing size and complexity of scientific data could dramatically enhance discovery and prediction for basic scientific applications. Realizing this potential, however, requ…
Sub-sampled Newton Methods with Non-uniform Sampling
Peng Xu, Jiyan Yang, Farbod Roosta-Khorasani +2
We consider the problem of finding the minimizer of a convex function of the form where a low-rank facto…
FLAG n' FLARE: Fast Linearly-Coupled Adaptive Gradient Methods
Xiang Cheng, Farbod Roosta-Khorasani, Stefan Palombo +2
We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions. We present accelera…
Sub-Sampled Newton Methods II: Local Convergence Rates
Farbod Roosta-Khorasani, Michael W. Mahoney
Many data-fitting applications require the solution of an optimization problem involving a sum of large number of functions of high dimensional parameter. Here, we consider the pro…
Sub-Sampled Newton Methods I: Globally Convergent Algorithms
Farbod Roosta-Khorasani, Michael W. Mahoney
Large scale optimization problems are ubiquitous in machine learning and data analysis and there is a plethora of algorithms for solving such problems. Many of these algorithms emp…
Variational Perspective on Local Graph Clustering
Kimon Fountoulakis, Farbod Roosta-Khorasan, Julian Shun +2
Modern graph clustering applications require the analysis of large graphs and this can be computationally expensive. In this regard, local spectral graph clustering methods aim to…
Schur properties of convolutions of gamma random variables
Farbod Roosta-Khorasani, Gabor J. Szekely
Sufficient conditions for comparing the convolutions of heterogeneous gamma random variables in terms of the usual stochastic order are established. Such comparisons are characteri…
Algorithms that satisfy a stopping criterion, probably
Uri Ascher, Farbod Roosta-Khorasani
Iterative numerical algorithms are typically equipped with a stopping criterion, where the iteration process is terminated when some error or misfit measure is deemed to be below a…
Assessing stochastic algorithms for large scale nonlinear least squares problems using extremal probabilities of linear combinations of gamma random variables
Farbod Roosta-Khorasani, Gábor J. Székely, Uri Ascher
This article considers stochastic algorithms for efficiently solving a class of large scale non-linear least squares (NLS) problems which frequently arise in applications. We propo…
Data completion and stochastic algorithms for PDE inversion problems with many measurements
Farbod Roosta-Khorasani, Kees van den Doel, Uri Ascher
Inverse problems involving systems of partial differential equations (PDEs) with many measurements or experiments can be very expensive to solve numerically. In a recent paper we e…
Improved bounds on sample size for implicit matrix trace estimators
Farbod Roosta-Khorasani, Uri Ascher
This article is concerned with Monte-Carlo methods for the estimation of the trace of an implicitly given matrix whose information is only available through matrix-vector produ…