Categorical semantics of a simple differential programming language
arXiv:2101.10491 · doi:10.4204/EPTCS.333.20
Abstract
With the increased interest in machine learning, and deep learning in particular, the use of automatic differentiation has become more wide-spread in computation. There have been two recent developments to provide the theoretical support for this types of structure. One approach, due to Abadi and Plotkin, provides a simple differential programming language. Another approach is the notion of a reverse differential category. In the present paper we bring these two approaches together. In particular, we show how an extension of reverse derivative categories models Abadi and Plotkin's language, and describe how this categorical model allows one to consider potential improvements to the operational semantics of the language.
arXiv commentIn Proceedings ACT 2020, arXiv:2101.07888
References in corpus (6)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Restriction categories III: colimits, partial limits, and extensivity
- An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
- A Simple Differentiable Programming Language
- Tangent: Automatic differentiation using source-code transformation for dynamically typed array programming
- Differential bundles and fibrations for tangent categories