704 citations · 4.8k across the 117 of their papers we have counts for
11 papers · 1 filter
Efficient Greedy Coordinate Descent for Composite Problems
Sai Praneeth Karimireddy, Anastasia Koloskova, Sebastian U. Stich +1
Coordinate descent with random coordinate selection is the current state of the art for many large scale optimization problems. However, greedy selection of the steepest coordinate…
Sparsified SGD with Memory
Sebastian U. Stich, Jean-Baptiste Cordonnier, Martin Jaggi
Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e. algorithms that leverage the compute power of many devices for training. The…
Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations
Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut +1
We present a framework for building unsupervised representations of entities and their compositions, where each entity is viewed as a probability distribution rather than a vector…
Don't Use Large Mini-Batches, Use Local SGD
Tao Lin, Sebastian U. Stich, Kumar Kshitij Patel +1
Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key effic…
A Potential Space Estimate for Solutions of Systems of Nonlocal Equations in Peridynamics
James Scott, Tadele Mengesha
We show that weak solutions to the strongly-coupled system of nonlocal equations of linearized peridynamics belong to a potential space with higher integrability. Specifically, we…
A Distributed Second-Order Algorithm You Can Trust
Celestine Dünner, Aurelien Lucchi, Matilde Gargiani +3
Due to the rapid growth of data and computational resources, distributed optimization has become an active research area in recent years. While first-order methods seem to dominate…
Linearly Convergent Frank-Wolfe with Backtracking Line-Search
Fabian Pedregosa, Geoffrey Negiar, Armin Askari +1
Structured constraints in Machine Learning have recently brought the Frank-Wolfe (FW) family of algorithms back in the spotlight. While the classical FW algorithm has poor local co…
Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients
Sai Praneeth Karimireddy, Sebastian U. Stich, Martin Jaggi
We show that Newton's method converges globally at a linear rate for objective functions whose Hessians are stable. This class of problems includes many functions which are not str…
Training DNNs with Hybrid Block Floating Point
Mario Drumond, Tao Lin, Martin Jaggi +1
The wide adoption of DNNs has given birth to unrelenting computing requirements, forcing datacenter operators to adopt domain-specific accelerators to train them. These accelerator…
On Matching Pursuit and Coordinate Descent
Francesco Locatello, Anant Raj, Sai Praneeth Karimireddy +4
Two popular examples of first-order optimization methods over linear spaces are coordinate descent and matching pursuit algorithms, with their randomized variants. While the former…
Simple Unsupervised Keyphrase Extraction using Sentence Embeddings
Kamil Bennani-Smires, Claudiu Musat, Andreea Hossmann +2
Keyphrase extraction is the task of automatically selecting a small set of phrases that best describe a given free text document. Supervised keyphrase extraction requires large amo…