2 citations · 2 across the 6 of their papers we have counts for
8 papers
Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts
Kwanhee Lee, Namhoon Lee, Dan Alistarh
Mixture-of-Experts (MoE) architectures allow frontier language models to scale to trillions of parameters, but their deployment is constrained by massive memory footprints and memo…
Exploiting Local Flatness for Efficient Out-of-Distribution Detection
Seonghwan Park, Hyunji Jung, Dongyeop Lee +1
Detecting out-of-distribution (OOD) data is crucial for reliable machine learning deployment. Among detection strategies, post-hoc methods are particularly attractive due to their…
Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence
Jinseok Chung, Minkyoung Song, Hyunji Jung +1
In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design…
The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM
Kwanhee Lee, Hyeondo Jang, Dongyeop Lee +2
Neural network pruning is a promising technique to mitigate the excessive computational and memory requirements of large language models (LLMs). Despite its promise, however, progr…
SAFE: Finding Sparse and Flat Minima to Improve Pruning
Dongyeop Lee, Kwanhee Lee, Jinseok Chung +1
Sparsifying neural networks often suffers from seemingly inevitable performance degradation, and it remains challenging to restore the original performance despite much recent prog…
An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations
Seonghwan Park, Jueun Mun, Donghyun Oh +1
Concept bottleneck models (CBMs) ensure interpretability by decomposing predictions into human interpretable concepts. Yet the annotations used for training CBMs that enable this t…
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation
Dahun Shin, Dongyeop Lee, Jinseok Chung +1
Approximate second-order optimization methods often exhibit poorer generalization compared to first-order approaches. In this work, we look into this issue through the lens of the…
FedFwd: Federated Learning without Backpropagation
Seonghwan Park, Dahun Shin, Jinseok Chung +1
In federated learning (FL), clients with limited resources can disrupt the training efficiency. A potential solution to this problem is to leverage a new learning procedure that do…