7 papers · 1 filter
RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization
Shenyang Deng, Zhuoli Ouyang, Tianyu Pang +4
Preconditioned adaptive methods have gained significant attention for training deep neural networks, as they capture rich curvature information of the loss landscape. The central c…
Landscaper: Understanding Loss Landscapes Through Multi-Dimensional Topological Analysis
Jiaqing Chen, Nicholas Hadler, Tiankai Xie +8
Loss landscapes are a powerful tool for understanding neural network optimization and generalization, yet traditional low-dimensional analyses often miss complex topological featur…
Learning to Discover Iterative Spectral Algorithms
Zihang Liu, Oleg Balabanov, Yaoqing Yang +1
We introduce AutoSpec, a neural network framework for discovering iterative spectral algorithms for large-scale numerical linear algebra and numerical optimization. Our self-superv…
The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators
Mansi Sakarvadia, Kareem Hegazy, Amin Totounferoush +4
A core challenge in scientific machine learning, and scientific computing more generally, is modeling continuous phenomena which (in practice) are represented discretely. Machine-l…
LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics
Tiankai Xie, Jiaqing Chen, Yaoqing Yang +8
Modern machine learning often relies on optimizing a neural network's parameters using a loss function to learn complex features. Beyond training, examining the loss function with…
Visualizing Loss Functions as Topological Landscape Profiles
Caleb Geniesse, Jiaqing Chen, Tiankai Xie +7
In machine learning, a loss function measures the difference between model predictions and ground-truth (or target) values. For neural network models, visualizing how this loss cha…