23 citations · 121 across the 23 of their papers we have counts for
4 papers · 1 filter
MoDeGPT: Modular Decomposition for Large Language Model Compression
Chi-Heng Lin, Shangqian Gao, James Seale Smith +5
Large Language Models (LLMs) have reshaped the landscape of artificial intelligence by demonstrating exceptional performance across various tasks. However, substantial computationa…
Language model compression with weighted low-rank factorization
Yen-Chang Hsu, Ting Hua, Sungen Chang +3
Factorizing a large matrix into small matrices is a popular strategy for model compression. Singular value decomposition (SVD) plays a vital role in this compression strategy, appr…
A Closer Look at Knowledge Distillation with Features, Logits, and Gradients
Yen-Chang Hsu, James Smith, Yilin Shen +2
Knowledge distillation (KD) is a substantial strategy for transferring learned knowledge from one neural network model to another. A vast number of methods have been developed for…
Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data
Junjiao Tian, Yen-Change Hsu, Yilin Shen +2
Moving beyond testing on in-distribution data works on Out-of-Distribution (OOD) detection have recently increased in popularity. A recent attempt to categorize OOD data introduces…