56 citations · 283 across the 30 of their papers we have counts for
5 papers · 1 filter
Adaptive and Implicit Regularization for Matrix Completion
Zhemin Li, Tao Sun, Hongxia Wang +1
The explicit low-rank regularization, e.g., nuclear norm regularization, has been widely used in imaging sciences. However, it has been found that implicit regularization outperfor…
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization
Tan Nguyen, Richard G. Baraniuk, Robert M. Kirby +2
Transformers have achieved remarkable success in sequence modeling and beyond but suffer from quadratic computational and memory complexities with respect to the length of the inpu…
Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs
Justin Baker, Hedi Xia, Yiwei Wang +7
Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers. These solvers are computationally expensive, requiring…
Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs
Justin Baker, Elena Cherkaev, Akil Narayan +1
Proper orthogonal decomposition (POD) allows reduced-order modeling of complex dynamical systems at a substantial level, while maintaining a high degree of accuracy in modeling the…
glassoformer: a query-sparse transformer for post-fault power grid voltage prediction
Yunling Zheng, Carson Hu, Guang Lin +3
We propose GLassoformer, a novel and efficient transformer architecture leveraging group Lasso regularization to reduce the number of queries of the standard self-attention mechani…