82 citations · 92 across the 7 of their papers we have counts for
8 papers
Optimizing Neural Networks through Activation Function Discovery and Automatic Weight Initialization
Garrett Bingham
Automated machine learning (AutoML) methods improve upon existing models by optimizing various aspects of their design. While present methods focus on hyperparameters and neural ne…
Efficient Activation Function Optimization through Surrogate Modeling
Garrett Bingham, Risto Miikkulainen
Carefully designed activation functions can improve the performance of neural networks in many machine learning tasks. However, it is difficult for humans to construct optimal acti…
AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks
Garrett Bingham, Risto Miikkulainen
Neural networks require careful weight initialization to prevent signals from exploding or vanishing. Existing initialization schemes solve this problem in specific cases by assumi…
Discovering Parametric Activation Functions
Garrett Bingham, Risto Miikkulainen
Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks. However, the benefits of novel activation funct…
Evolutionary Optimization of Deep Learning Activation Functions
Garrett Bingham, William Macke, Risto Miikkulainen
The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, t…
Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations
Rui Zhang, Caitlin Westerfield, Sungrok Shim +5
In this paper, we propose to boost low-resource cross-lingual document retrieval performance with deep bilingual query-document representations. We match queries and documents in b…