29 citations · 35 across the 4 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.LG2020★ 29 cited
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan +8
Transformer models have achieved state-of-the-art results across a diverse range of domains. However, concern over the cost of training the attention mechanism to learn complex dep…
cs.LG2020★ 4 cited
Orthogonal Over-Parameterized Training
Weiyang Liu, Rongmei Lin, Zhen Liu +5
The inductive bias of a neural network is largely determined by the architecture and the training algorithm. To achieve good generalization, how to effectively train a neural netwo…