3 citations · 5 across the 4 of their papers we have counts for
4 papers
Spark Transformer: Reactivating Sparsity in FFN and Attention
Chong You, Kan Wu, Zhipeng Jia +16
The discovery of the lazy neuron phenomenon in trained Transformers, where the vast majority of neurons in their feed-forward networks (FFN) are inactive for each token, has spurre…
Scalable Machine Learning Training Infrastructure for Online Ads Recommendation and Auction Scoring Modeling at Google
George Kurian, Somayeh Sardashti, Ryan Sims +8
Large-scale Ads recommendation and auction scoring models at Google scale demand immense computational resources. While specialized hardware like TPUs have improved linear algebra…
The Power of Discrete Quantum Theories
Andrew J. Hanson, Gerardo Ortiz, Amr Sabry +1
We explore the implications of restricting the framework of quantum theory and quantum computation to finite fields. The simplest proposed theory is defined over arbitrary finite f…
Solving UNIQUE-SAT in a Modal Quantum Theory
Jeremiah Willcock, Amr Sabry
In recent work, Benjamin Schumacher and Michael D. Westmoreland investigate a version of quantum mechanics which they call modal quantum theory. This theory is obtained by instanti…