267 citations · 281 across the 3 of their papers we have counts for
3 papers
cs.LG2017★ 267 cited
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz +4
The capacity of a neural network to absorb information is limited by its number of parameters. Conditional computation, where parts of the network are active on a per-example basis…
cs.LG2014★ 11 cited
Skip-gram Language Modeling Using Sparse Non-negative Matrix Probability Estimation
Noam Shazeer, Joris Pelemans, Ciprian Chelba
We present a novel family of language model (LM) estimation techniques named Sparse Non-negative Matrix (SNM) estimation. A first set of experiments empirically evaluating it on th…
cs.AI2010★ 3 cited
Variational Program Inference
Georges Harik, Noam Shazeer
We introduce a framework for representing a variety of interesting problems as inference over the execution of probabilistic model programs. We represent a "solution" to such a pro…