374 citations · 941 across the 27 of their papers we have counts for
7 papers · 1 filter
OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition
Stephen Zhang, Vardan Papyan
The recent paradigm shift to large-scale foundation models has brought about a new era for deep learning that, while has found great success in practice, has also been plagued by p…
Sparsest Models Elude Pruning: An Exposé of Pruning's Current Capabilities
Stephen Zhang, Vardan Papyan
Pruning has emerged as a promising approach for compressing large-scale models, yet its effectiveness in recovering the sparsest of models has not yet been explored. We conducted a…
Transformer Block Coupling and its Correlation with Generalization in LLMs
Murdock Aubry, Haoming Meng, Anton Sugolov +1
Large Language Models (LLMs) have made significant strides in natural language processing, and a precise understanding of the internal mechanisms driving their success is essential…
Breach By A Thousand Leaks: Unsafe Information Leakage in `Safe' AI Responses
David Glukhov, Ziwen Han, Ilia Shumailov +2
Vulnerability of Frontier language models to misuse and jailbreaks has prompted the development of safety measures like filters and alignment training in an effort to ensure safety…
Linguistic Collapse: Neural Collapse in (Large) Language Models
Robert Wu, Vardan Papyan
Neural collapse () is a phenomenon observed in classification tasks where top-layer representations collapse into their class means, which become equinorm, equiangula…
Pushing Boundaries: Mixup's Influence on Neural Collapse
Quinn Fisher, Haoming Meng, Vardan Papyan
Mixup is a data augmentation strategy that employs convex combinations of training instances and their respective labels to augment the robustness and calibration of deep neural ne…