32 citations · 32 across the 9 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization
Xinhao Yao, Hongjin Qian, Xiaolin Hu +5
Large Language Models (LLMs), built on Transformer architectures, exhibit remarkable generalization across a wide range of tasks. However, fine-tuning these models for specific tas…
cs.LG2024
Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective
Xinhao Yao, Xiaolin Hu, Shenzhi Yang +1
Pre-trained large language models (LLMs) based on Transformer have demonstrated striking in-context learning (ICL) abilities. With a few demonstration input-label pairs, they can p…