108 citations · 353 across the 17 of their papers we have counts for
7 papers · 1 filter
Transformer with Memory Replay
Rui Liu, Barzan Mozafari
Transformers achieve state-of-the-art performance for natural language processing tasks by pre-training on large-scale text corpora. They are extremely compute-intensive and have v…
Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers
Rui Liu, Young Jin Kim, Alexandre Muzio +1
Sparsely activated transformers, such as Mixture of Experts (MoE), have received great interest due to their outrageous scaling capability which enables dramatical increases in mod…
Adam with Bandit Sampling for Deep Learning
Rui Liu, Tianyi Wu, Barzan Mozafari
Adam is a widely used optimization method for training deep learning models. It computes individual adaptive learning rates for different parameters. In this paper, we propose a ge…
BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees
Yongjoo Park, Jingyi Qing, Xiaoyang Shen +1
The rising volume of datasets has made training machine learning (ML) models a major computational cost in the enterprise. Given the iterative nature of model and parameter tuning,…
A Bandit Approach to Maximum Inner Product Search
Rui Liu, Tianyi Wu, Barzan Mozafari
There has been substantial research on sub-linear time approximate algorithms for Maximum Inner Product Search (MIPS). To achieve fast query time, state-of-the-art techniques requi…
Revisiting Projection-Free Optimization for Strongly Convex Constraint Sets
Jarrid Rector-Brooks, Jun-Kun Wang, Barzan Mozafari
We revisit the Frank-Wolfe (FW) optimization under strongly convex constraint sets. We provide a faster convergence rate for FW without line search, showing that a previously overl…
Active Learning for Crowd-Sourced Databases
Barzan Mozafari, Purnamrita Sarkar, Michael J. Franklin +2
Crowd-sourcing has become a popular means of acquiring labeled data for a wide variety of tasks where humans are more accurate than computers, e.g., labeling images, matching objec…