Enhancing elusive clues in knowledge learning by contrasting attention of language models

Jian Gao, Xiao Zhang, Ji Wu +1

Causal language models acquire vast amount of knowledge from general text corpus during pretraining, but the efficiency of knowledge learning is known to be unsatisfactory, especia…