704 citations · 4.8k across the 117 of their papers we have counts for
19 papers · 1 filter
Robust Cross-lingual Embeddings from Parallel Sentences
Ali Sabet, Prakhar Gupta, Jean-Baptiste Cordonnier +2
Recent advances in cross-lingual word embeddings have primarily relied on mapping-based methods, which project pretrained word embeddings from different languages into a shared spa…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
On the Relationship between Self-Attention and Convolutional Layers
Jean-Baptiste Cordonnier, Andreas Loukas, Martin Jaggi
Recent trends of incorporating attention mechanisms in vision have led researchers to reconsider the supremacy of convolutional layers as a primary building block. Beyond helping C…
Optimizer Benchmarking Needs to Account for Hyperparameter Tuning
Prabhu Teja Sivaprasad, Florian Mai, Thijs Vogels +2
The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration. The efficacy of optimizers is often studied under n…
Model Fusion via Optimal Transport
Sidak Pal Singh, Martin Jaggi
Combining different models is a widely used paradigm in machine learning applications. While the most common approach is to form an ensemble of models and average their individual…
Decentralized Deep Learning with Arbitrary Communication Compression
Anastasia Koloskova, Tao Lin, Sebastian U. Stich +1
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute cl…
Correlating Twitter Language with Community-Level Health Outcomes
Arno Schneuwly, Ralf Grubenmann, Séverine Rion Logean +2
We study how language on social media is linked to diseases such as atherosclerotic heart disease (AHD), diabetes and various types of cancer. Our proposed model leverages state-of…
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
Thijs Vogels, Sai Praneeth Karimireddy, Martin Jaggi
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization. Despite the significant attention received, current compr…
On Linear Learning with Manycore Processors
Eliza Wszola, Celestine Mendler-Dünner, Martin Jaggi +1
A new generation of manycore processors is on the rise that offers dozens and more cores on a chip and, in a sense, fuses host processor and accelerator. In this paper we target th…
Better Word Embeddings by Disentangling Contextual n-Gram Information
Prakhar Gupta, Matteo Pagliardini, Martin Jaggi
Pre-trained word vectors are ubiquitous in Natural Language Processing applications. In this paper, we show how training word embeddings jointly with bigram and even trigram embedd…
Crosslingual Document Embedding as Reduced-Rank Ridge Regression
Martin Josifoski, Ivan S. Paskov, Hristo S. Paskov +2
There has recently been much interest in extending vector-based word representations to multiple languages, such that words can be compared across languages. In this paper, we shif…
MLSys: The New Frontier of Machine Learning Systems
Alexander Ratner, Dan Alistarh, Gustavo Alonso +66
Machine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and implementing the systems that support ML models in real-world deployments remains…
Overcoming Multi-Model Forgetting
Yassine Benyahia, Kaicheng Yu, Kamil Bennani-Smires +4
We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performan…
Interpretable Structure-aware Document Encoders with Hierarchical Attention
Khalil Mrini, Claudiu Musat, Michael Baeriswyl +1
We propose a method to create document representations that reflect their internal structure. We modify Tree-LSTMs to hierarchically merge basic elements such as words and sentence…
Forecasting intracranial hypertension using multi-scale waveform metrics
Matthias Hüser, Adrian Kündig, Walter Karlen +2
Objective: Acute intracranial hypertension is an important risk factor of secondary brain damage after traumatic brain injury. Hypertensive episodes are often diagnosed reactively,…
Evaluating the Search Phase of Neural Architecture Search
Kaicheng Yu, Christian Sciuto, Martin Jaggi +2
Neural Architecture Search (NAS) aims to facilitate the design of deep networks for new tasks. Existing techniques rely on two stages: searching over the architecture space and val…
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication
Anastasia Koloskova, Sebastian U. Stich, Martin Jaggi
We consider decentralized stochastic optimization with the objective function (e.g. data samples for machine learning task) being distributed over machines that can only commun…
Unsupervised Scalable Representation Learning for Multivariate Time Series
Jean-Yves Franceschi, Aymeric Dieuleveut, Martin Jaggi
Time series constitute a challenging data type for machine learning algorithms, due to their highly variable lengths and sparse labeling in practice. In this paper, we tackle this…
Error Feedback Fixes SignSGD and other Gradient Compression Schemes
Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian U. Stich +1
Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks acros…