From the 1 of 11 linked papers with an AI index.
7 papers · 1 filter
Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations
Victor M. Yeom-Song, Severi Rissanen, Arno Solin +2
Diffusion models have become a powerful generative prior for solutions of partial differential equations (PDEs). Existing approaches enforce physical constraints either by adding t…
Convergent Stochastic Training of Attention and Understanding LoRA
Zhengkai Sun, Dibyakanti Kumar, Alejandro F Frangi +2
Transformers have revolutionized machine learning and deploying attention layers in the model is increasingly standard across a myriad of applications. Further, for large models, i…
Gradient Regularized Natural Gradients
Satya Prakash Dash, Hossein Abdi, Wei Pan +2
Gradient regularization (GR) has been shown to improve the generalizability of trained models. While Natural Gradient Descent has been shown to accelerate optimization in the initi…
On the Generalization Behavior of Deep Residual Networks From a Dynamical System Perspective
Jinshu Huang, Mingfei Sun, Chunlin Wu
Deep neural networks (DNNs) have significantly advanced machine learning, with model depth playing a central role in their successes. The dynamical system modeling approach has rec…
Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learning
Yingxiao Huo, Satya Prakash Dash, Radu Stoican +2
Natural gradients have long been studied in deep reinforcement learning due to their fast convergence properties and covariant weight updates. However, computing natural gradients…
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models
Hossein Abdi, Mingfei Sun, Andi Zhang +2
Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly…