39 citations · 89 across the 16 of their papers we have counts for
17 papers
The Foil: Capture-Avoiding Substitution With No Sharp Edges
Dougal Maclaurin, Alexey Radul, Adam Paszke
Correctly manipulating program terms in a compiler is surprisingly difficult because of the need to avoid name capture. The rapier from "Secrets of the Glasgow Haskell Compiler inl…
You Only Linearize Once: Tangents Transpose to Gradients
Alexey Radul, Adam Paszke, Roy Frostig +2
Automatic differentiation (AD) is conventionally understood as a family of distinct algorithms, rooted in two "modes" -- forward and reverse -- which are typically presented (and i…
Decomposing reverse-mode automatic differentiation
Roy Frostig, Matthew J. Johnson, Dougal Maclaurin +2
We decompose reverse-mode automatic differentiation into (forward-mode) linearization followed by transposition. Doing so isolates the essential difference between forward- and rev…
Getting to the Point. Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programming
Adam Paszke, Daniel Johnson, David Duvenaud +5
We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level n…
The Base Measure Problem and its Solution
Alexey Radul, Boris Alexeev
Probabilistic programming systems generally compute with probability density functions, leaving the base measure of each such function implicit. This mostly works, but creates prob…
FunMC: A functional API for building Markov Chains
Pavel Sountsov, Alexey Radul, Srinivas Vasudevan
Constant-memory algorithms, also loosely called Markov chains, power the vast majority of probabilistic inference and machine learning applications today. A lot of progress has bee…
Automatically Batching Control-Intensive Programs for Modern Accelerators
Alexey Radul, Brian Patton, Dougal Maclaurin +2
We present a general approach to batching arbitrary computations for accelerators such as GPUs. We show orders-of-magnitude speedups using our method on the No U-Turn Sampler (NUTS…
Simple, Distributed, and Accelerated Probabilistic Programming
Dustin Tran, Matthew Hoffman, Dave Moore +5
We describe a simple, low-level approach for embedding probabilistic programming in a deep learning ecosystem. In particular, we distill probabilistic programming down to a single…
Probabilistic programs for inferring the goals of autonomous agents
Marco F. Cusumano-Towner, Alexey Radul, David Wingate +1
Intelligent systems sometimes need to infer the probable goals of people, cars, and robots, based on partial observations of their motion. This paper introduces a class of probabil…
Time Series Structure Discovery via Probabilistic Program Synthesis
Ulrich Schaechtle, Feras Saad, Alexey Radul +1
There is a widespread need for techniques that can discover structure from time series data. Recently introduced techniques such as Automatic Bayesian Covariance Discovery (ABCD) p…
Probabilistic Programming with Gaussian Process Memoization
Ulrich Schaechtle, Ben Zinberg, Alexey Radul +2
Gaussian Processes (GPs) are widely used tools in statistics, machine learning, robotics, computer vision, and scientific computation. However, despite their popularity, they can b…
Automatic differentiation in machine learning: a survey
Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul +1
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "au…
Confusion of Tagged Perturbations in Forward Automatic Differentiation of Higher-Order Functions
Oleksandr Manzyuk, Barak A. Pearlmutter, Alexey Andreyevich Radul +2
Forward Automatic Differentiation (AD) is a technique for augmenting programs to compute derivatives. The essence of Forward AD is to attach perturbations to each number, and propa…
AD in Fortran, Part 2: Implementation via Prepreprocessor
Alexey Radul, Barak A. Pearlmutter, Jeffrey Mark Siskind
We describe an implementation of the Farfel Fortran AD extensions. These extensions integrate forward and reverse AD directly into the programming model, with attendant benefits to…
AD in Fortran, Part 1: Design
Alexey Radul, Barak A. Pearlmutter, Jeffrey Mark Siskind
We propose extensions to Fortran which integrate forward and reverse Automatic Differentiation (AD) directly into the programming model. Irrespective of implementation technology,…
Jewish Problems
Tanya Khovanova, Alexey Radul
This is a special collection of problems that were given to select applicants during oral entrance exams to the math department of Moscow State University. These problems were desi…
Baron Munchhausen's Sequence
Tanya Khovanova, Konstantin Knop, Alexey Radul
We investigate a coin-weighing puzzle that appeared in the all-Russian math Olympiad in 2000. We liked the puzzle because the methods of analysis differ from classical coin-weighin…