A Framework for Quantifying How Pre-Training and Context Benefit In-Context Learning

Bingqing Song, Jiaxiang Li, Rong Wang +2

Pre-trained large language models have demonstrated a strong ability to learn from context, known as in-context learning (ICL). Despite a surge of recent applications that leverage…