2 papers
cs.AI2024
Enhancing elusive clues in knowledge learning by contrasting attention of language models
Jian Gao, Xiao Zhang, Ji Wu +1
Causal language models acquire vast amount of knowledge from general text corpus during pretraining, but the efficiency of knowledge learning is known to be unsatisfactory, especia…
cs.CL2024
Co-occurrence is not Factual Association in Language Models
Xiao Zhang, Miao Li, Ji Wu
Pretrained language models can encode a large amount of knowledge and utilize it for various reasoning tasks, yet they can still struggle to learn novel factual knowledge effective…