35 citations · 67 across the 9 of their papers we have counts for
7 papers · 1 filter
Balanced and Deterministic Weight-sharing Helps Network Performance
Oscar Chang, Hod Lipson
Weight-sharing plays a significant role in the success of many deep neural networks, by increasing memory efficiency and incorporating useful inductive priors about the problem int…
Principled Weight Initialization for Hypernetworks
Oscar Chang, Lampros Flokas, Hod Lipson
Hypernetworks are meta neural networks that generate weights for a main neural network in an end-to-end differentiable manner. Despite extensive applications ranging from multi-tas…
Accelerating Meta-Learning by Sharing Gradients
Oscar Chang, Hod Lipson
The success of gradient-based meta-learning is primarily attributed to its ability to leverage related tasks to learn task-invariant information. However, the absence of interactio…
Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network
Oscar Chang, Yuling Yao, David Williams-King +1
Two main obstacles preventing the widespread adoption of variational Bayesian neural networks are the high parameter overhead that makes them infeasible on large networks, and the…
Seven Myths in Machine Learning Research
Oscar Chang, Hod Lipson
We present seven myths commonly believed to be true in machine learning research, circa Feb 2019. This is an archival copy of the blog post at https://crazyoscarchang.github.io/201…
Agent Embeddings: A Latent Representation for Pole-Balancing Networks
Oscar Chang, Robert Kwiatkowski, Siyuan Chen +1
We show that it is possible to reduce a high-dimensional object like a neural network agent into a low-dimensional vector representation with semantic meaning that we call agent em…
Gradient Normalization & Depth Based Decay For Deep Learning
Robert Kwiatkowski, Oscar Chang
In this paper we introduce a novel method of gradient normalization and decay with respect to depth. Our method leverages the simple concept of normalizing all gradients in a deep…