Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models

You Li, Jinhui Yin, Yuming Lin

Pretrained language models are expected to effectively map input text to a set of vectors while preserving the inherent relationships within the text. Consequently, designing a whi…