5 papers
Exploiting Local Flatness for Efficient Out-of-Distribution Detection
Seonghwan Park, Hyunji Jung, Dongyeop Lee +1
Detecting out-of-distribution (OOD) data is crucial for reliable machine learning deployment. Among detection strategies, post-hoc methods are particularly attractive due to their…
The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM
Kwanhee Lee, Hyeondo Jang, Dongyeop Lee +2
Neural network pruning is a promising technique to mitigate the excessive computational and memory requirements of large language models (LLMs). Despite its promise, however, progr…
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation
Dahun Shin, Dongyeop Lee, Jinseok Chung +1
Approximate second-order optimization methods often exhibit poorer generalization compared to first-order approaches. In this work, we look into this issue through the lens of the…
SAFE: Finding Sparse and Flat Minima to Improve Pruning
Dongyeop Lee, Kwanhee Lee, Jinseok Chung +1
Sparsifying neural networks often suffers from seemingly inevitable performance degradation, and it remains challenging to restore the original performance despite much recent prog…
Critical Influence of Overparameterization on Sharpness-aware Minimization
Sungbin Shin, Dongyeop Lee, Maksym Andriushchenko +1
Sharpness-Aware Minimization (SAM) has attracted considerable attention for its effectiveness in improving generalization in deep neural network training by explicitly minimizing s…