3 papers
cs.CL2025
Co-occurrence is not Factual Association in Language Models
Xiao Zhang, Miao Li, Ji Wu
Pretrained language models can encode a large amount of knowledge and utilize it for various reasoning tasks, yet they can still struggle to learn novel factual knowledge effective…
cs.AI2025
Enhancing elusive clues in knowledge learning by contrasting attention of language models
Jian Gao, Xiao Zhang, Ji Wu +1
Causal language models acquire vast amount of knowledge from general text corpus during pretraining, but the efficiency of knowledge learning is known to be unsatisfactory, especia…
cs.CL2024
Conditional Language Learning with Context
Xiao Zhang, Miao Li, Ji Wu
Language models can learn sophisticated language understanding skills from fitting raw text. They also unselectively learn useless corpus statistics and biases, especially during f…