Differentiate Everything with a Reversible Embeded Domain-Specific Language

Jin-Guo Liu, Taine Zhao

Reverse-mode automatic differentiation (AD) suffers from the issue of having too much space overhead to trace back intermediate computational states for back-propagation. The tradi…