Reconciling In-Context and In-Weight Learning via Dual Representation Space Encoding

Guanyu Chen, Ruichen Wang, Tianren Zhang +1

In-context learning (ICL) is a valuable capability exhibited by Transformers pretrained on diverse sequence tasks. However, previous studies have observed that ICL often conflicts…