5 papers
Landscaper: Understanding Loss Landscapes Through Multi-Dimensional Topological Analysis
Jiaqing Chen, Nicholas Hadler, Tiankai Xie +8
Loss landscapes are a powerful tool for understanding neural network optimization and generalization, yet traditional low-dimensional analyses often miss complex topological featur…
Visualizing Loss Functions as Topological Landscape Profiles
Caleb Geniesse, Jiaqing Chen, Tiankai Xie +7
In machine learning, a loss function measures the difference between model predictions and ground-truth (or target) values. For neural network models, visualizing how this loss cha…
Evaluating Loss Landscapes from a Topology Perspective
Tiankai Xie, Caleb Geniesse, Jiaqing Chen +5
Characterizing the loss of a neural network with respect to model parameters, i.e., the loss landscape, can provide valuable insights into properties of that model. Various methods…
AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
Haiquan Lu, Yefan Zhou, Shiwei Liu +3
Recent work on pruning large language models (LLMs) has shown that one can eliminate a large number of parameters without compromising performance, making pruning a promising strat…
Mitigating Memorization In Language Models
Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6
Language models (LMs) can "memorize" information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that d…