187 citations · 529 across the 15 of their papers we have counts for
15 papers
Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation
Xuesu Xiao, Tingnan Zhang, Krzysztof Choromanski +14
Despite decades of research, existing navigation systems still face real-world challenges when deployed in the wild, e.g., in cluttered home environments or in human-occupied publi…
You Only Linearize Once: Tangents Transpose to Gradients
Alexey Radul, Adam Paszke, Roy Frostig +2
Automatic differentiation (AD) is conventionally understood as a family of distinct algorithms, rooted in two "modes" -- forward and reverse -- which are typically presented (and i…
Learning from many trajectories
Stephen Tu, Roy Frostig, Mahdi Soltanolkotabi
We initiate a study of supervised learning from many independent sequences ("trajectories") of non-independent covariates, reflecting tasks in sequence modeling, control, and reinf…
Decomposing reverse-mode automatic differentiation
Roy Frostig, Matthew J. Johnson, Dougal Maclaurin +2
We decompose reverse-mode automatic differentiation into (forward-mode) linearization followed by transposition. Doing so isolates the essential difference between forward- and rev…
Efficient and Modular Implicit Differentiation
Mathieu Blondel, Quentin Berthet, Marco Cuturi +5
Automatic differentiation (autodiff) has revolutionized machine learning. It allows to express complex computations by composing elementary ones in creative ways and removes the bu…
The advantages of multiple classes for reducing overfitting from test set reuse
Vitaly Feldman, Roy Frostig, Moritz Hardt
Excessive reuse of holdout data can lead to overfitting. However, there is little concrete evidence of significant overfitting due to holdout reuse in popular multiclass benchmarks…
Measuring the Effects of Data Parallelism on Neural Network Training
Christopher J. Shallue, Jaehoon Lee, Joseph Antognini +3
Recent hardware developments have dramatically increased the scale of data parallelism available for neural network training. Among the simplest ways to harness next-generation har…
Random Features for Compositional Kernels
Amit Daniely, Roy Frostig, Vineet Gupta +1
We describe and analyze a simple random feature scheme (RFS) from prescribed compositional kernels. The compositional kernels we use are inspired by the structure of convolutional…
Estimation from Indirect Supervision with Linear Moments
Aditi Raghunathan, Roy Frostig, John Duchi +1
In structured prediction problems where we have indirect supervision of the output, maximum marginal likelihood faces two computational obstacles: non-convexity of the objective an…
Principal Component Projection Without Principal Component Analysis
Roy Frostig, Cameron Musco, Christopher Musco +1
We show how to efficiently project a vector onto the top principal components of a matrix, without explicitly computing these components. Specifically, we introduce an iterative al…
Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity
Amit Daniely, Roy Frostig, Yoram Singer
We develop a general duality between neural networks and compositional kernels, striving towards a better understanding of deep learning. We show that initial representations gener…
Un-regularizing: approximate proximal point and faster stochastic algorithms for empirical risk minimization
Roy Frostig, Rong Ge, Sham M. Kakade +1
We develop a family of accelerated stochastic algorithms that minimize sums of convex functions. Our algorithms improve upon the fastest running time for empirical risk minimizatio…
Competing with the Empirical Risk Minimizer in a Single Pass
Roy Frostig, Rong Ge, Sham M. Kakade +1
In many estimation problems, e.g. linear and logistic regression, we wish to minimize an unknown objective given only unbiased samples of the objective function. Furthermore, we ai…
A sub-constant improvement in approximating the positive semidefinite Grothendieck problem
Roy Frostig, Sida I. Wang
Semidefinite relaxations are a powerful tool for approximately solving combinatorial optimization problems such as MAX-CUT and the Grothendieck problem. By exploiting a bounded ran…
Relaxations for inference in restricted Boltzmann machines
Sida I. Wang, Roy Frostig, Percy Liang +1
We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in…