269 citations · 410 across the 21 of their papers we have counts for
16 papers · 1 filter
Linear Transformer Topological Masking with Graph Random Features
Isaac Reid, Kumar Avinava Dubey, Deepali Jain +12
When training transformers on graph-structured data, incorporating information about the underlying topology is crucial for good performance. Topological masking, a type of relativ…
When in Doubt, Summon the Titans: Efficient Inference with Large Models
Ankit Singh Rawat, Manzil Zaheer, Aditya Krishna Menon +2
Scaling neural networks to "large" sizes, with billions of parameters, has been shown to yield impressive results on many challenging problems. However, the inference cost incurred…
Hierarchically Regularized Deep Forecasting
Biswajit Paria, Rajat Sen, Amr Ahmed +1
Hierarchical forecasting is a key problem in many practical multivariate forecasting applications - the goal is to simultaneously predict a large number of correlated time series t…
Exact and Approximate Hierarchical Clustering Using A*
Craig S. Greenberg, Sebastian Macaluso, Nicholas Monath +6
Hierarchical clustering is a critical task in numerous domains. Many approaches are based on heuristics and the properties of the resulting clusterings are studied post hoc. Howeve…
Amazon SageMaker Autopilot: a white box AutoML solution at scale
Piali Das, Valerio Perrone, Nikita Ivkin +22
AutoML systems provide a black-box solution to machine learning problems by selecting the right way of processing features, choosing an algorithm and tuning the hyperparameters of…
Amazon SageMaker Automatic Model Tuning: Scalable Gradient-Free Optimization
Valerio Perrone, Huibin Shen, Aida Zolic +12
Tuning complex machine learning systems is challenging. Machine learning typically requires to set hyperparameters, be it regularization, architecture, or optimization parameters,…
Non-Stationary Latent Bandits
Joey Hong, Branislav Kveton, Manzil Zaheer +4
Users of recommender systems often behave in a non-stationary fashion, due to their evolving preferences and tastes over time. In this work, we propose a practical approach for fas…
Scalable Hierarchical Agglomerative Clustering
Nicholas Monath, Avinava Dubey, Guru Guruganesh +9
The applicability of agglomerative clustering, for inferring both hierarchical and flat clustering, is limited by its scalability. Existing scalable hierarchical clustering methods…
Big Bird: Transformers for Longer Sequences
Manzil Zaheer, Guru Guruganesh, Avinava Dubey +8
Transformers-based models, such as BERT, have been one of the most successful deep learning models for NLP. Unfortunately, one of their core limitations is the quadratic dependency…
Latent Bandits Revisited
Joey Hong, Branislav Kveton, Manzil Zaheer +3
A latent bandit problem is one in which the learning agent knows the arm reward distributions conditioned on an unknown discrete latent state. The primary goal of the agent is to i…
Non-Stationary Off-Policy Optimization
Joey Hong, Branislav Kveton, Manzil Zaheer +2
Off-policy learning is a framework for evaluating and optimizing policies without deploying them, from data collected by another policy. Real-world environments are typically non-s…
Anchor & Transform: Learning Sparse Embeddings for Large Vocabularies
Paul Pu Liang, Manzil Zaheer, Yuan Wang +1
Learning continuous representations of discrete objects such as text, users, movies, and URLs lies at the heart of many applications including language and user modeling. When usin…
State Space LSTM Models with Particle MCMC Inference
Xun Zheng, Manzil Zaheer, Amr Ahmed +3
Long Short-Term Memory (LSTM) is one of the most powerful sequence models. Despite the strong performance, however, it lacks the nice interpretability as in state space models. In…
Explaining reviews and ratings with PACO: Poisson Additive Co-Clustering
Chao-Yuan Wu, Alex Beutel, Amr Ahmed +1
Understanding a user's motivations provides valuable information beyond the ability to recommend items. Quite often this can be accomplished by perusing both ratings and review tex…
High Performance Latent Variable Models
Aaron Q. Li, Amr Ahmed, Mu Li +1
Latent variable models have accumulated a considerable amount of interest from the industry and academia for their versatility in a wide range of applications. A large amount of ef…
ACCAMS: Additive Co-Clustering to Approximate Matrices Succinctly
Alex Beutel, Amr Ahmed, Alexander J. Smola
Matrix completion and approximation are popular tools to capture a user's preferences for recommendation and to approximate missing data. Instead of using low-rank factorization we…