2 citations · 2 across the 9 of their papers we have counts for
9 papers
Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
Yuxin Wang, Yuanzhe Hu, Xiaokun Zhong +7
Neural networks trained under different hyperparameter settings can fall into distinct training "regimes," with consistent behavior within regimes and qualitative differences acros…
Landscaper: Understanding Loss Landscapes Through Multi-Dimensional Topological Analysis
Jiaqing Chen, Nicholas Hadler, Tiankai Xie +8
Loss landscapes are a powerful tool for understanding neural network optimization and generalization, yet traditional low-dimensional analyses often miss complex topological featur…
Learning to Discover Iterative Spectral Algorithms
Zihang Liu, Oleg Balabanov, Yaoqing Yang +1
We introduce AutoSpec, a neural network framework for discovering iterative spectral algorithms for large-scale numerical linear algebra and numerical optimization. Our self-superv…
Visualizing Loss Functions as Topological Landscape Profiles
Caleb Geniesse, Jiaqing Chen, Tiankai Xie +7
In machine learning, a loss function measures the difference between model predictions and ground-truth (or target) values. For neural network models, visualizing how this loss cha…
Evaluating Loss Landscapes from a Topology Perspective
Tiankai Xie, Caleb Geniesse, Jiaqing Chen +5
Characterizing the loss of a neural network with respect to model parameters, i.e., the loss landscape, can provide valuable insights into properties of that model. Various methods…
AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
Haiquan Lu, Yefan Zhou, Shiwei Liu +3
Recent work on pruning large language models (LLMs) has shown that one can eliminate a large number of parameters without compromising performance, making pruning a promising strat…
Mitigating Memorization In Language Models
Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6
Language models (LMs) can "memorize" information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that d…
Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
Haiquan Lu, Xiaotian Liu, Yefan Zhou +6
Recent studies on deep ensembles have identified the sharpness of the local minima of individual learners and the diversity of the ensemble members as key factors in improving test…
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
Yefan Zhou, Tianyu Pang, Keqin Liu +3
Regularization in modern machine learning is crucial, and it can take various forms in algorithmic design: training set, model family, error function, regularization terms, and opt…