On the Interplay Between Fine-tuning and Composition in Transformers

Lang Yu, Allyson Ettinger

Pre-trained transformer language models have shown remarkable performance on a variety of NLP tasks. However, recent research has suggested that phrase-level representations in the…