Language Model Decomposition: Quantifying the Dependency and Correlation of Language Models

Hao Zhang

Pre-trained language models (LMs), such as BERT (Devlin et al., 2018) and its variants, have led to significant improvements on various NLP tasks in past years. However, a theoreti…