Improving generalization in large language models by learning prefix subspaces

Louis Falissard, Vincent Guigue, Laure Soulier

This article focuses on large language models (LLMs) fine-tuning in the scarce data regime (also known as the "few-shot" learning setting). We propose a method to increase the gene…