4 papers
Improved denoising diffusion probabilistic models with efficient non-diagonal covariance modeling
Rui Xia, Ayan Das, Artem Artemev +3
The sampling process of Denoising Diffusion Probabilistic Models (DDPMs) can be accelerated by leveraging second-order information in the form of approximations to the denoising po…
Exploiting weight-space symmetries for approximating curvature
Artem Artemev, Rui Xia, Benjamin M. Boyd +4
Many machine learning techniques rely on approximating a loss function's curvature, but this is notoriously hard to do at the scale of modern deep networks. Surprisingly, no previo…
Efficient Model Compression Techniques with FishLeg
Jamie McGowan, Wei Sheng Lai, Weibin Chen +7
In many domains, the most successful AI models tend to be the largest, indeed often too large to be handled by AI players with limited computational resources. To mitigate this, a…
Exact, Tractable Gauss-Newton Optimization in Deep Reversible Architectures Reveal Poor Generalization
Davide Buffelli, Jamie McGowan, Wangkun Xu +4
Second-order optimization has been shown to accelerate the training of deep neural networks in many applications, often yielding faster progress per iteration on the training loss…