1 citations · 2 across the 3 of their papers we have counts for
5 papers · 1 filter
Training with Mixed-Precision Floating-Point Assignments
Wonyeol Lee, Rahul Sharma, Alex Aiken
When training deep neural networks, keeping all tensors in high precision (e.g., 32-bit or even 16-bit floats) is often wasteful. However, keeping all tensors in low precision (e.g…
On the Correctness of Automatic Differentiation for Neural Networks with Machine-Representable Parameters
Wonyeol Lee, Sejun Park, Alex Aiken
Recent work has shown that forward- and reverse- mode automatic differentiation (AD) over the reals is almost always correct in a mathematically precise sense. However, actual prog…
On Correctness of Automatic Differentiation for Non-Differentiable Functions
Wonyeol Lee, Hangyeol Yu, Xavier Rival +1
Differentiation lies at the core of many machine-learning algorithms, and is well-supported by popular autodiff systems, such as TensorFlow and PyTorch. Originally, these systems h…
Differentiable Algorithm for Marginalising Changepoints
Hyoungjin Lim, Gwonsoo Che, Wonyeol Lee +1
We present an algorithm for marginalising changepoints in time-series models that assume a fixed number of unknown changepoints. Our algorithm is differentiable with respect to its…
Reparameterization Gradient for Non-differentiable Models
Wonyeol Lee, Hangyeol Yu, Hongseok Yang
We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational infere…