Pretraining Language Models with Text-Attributed Heterogeneous Graphs

Tao Zou, Le Yu, Yifei Huang +2

In many real-world scenarios (e.g., academic networks, social platforms), different types of entities are not only associated with texts but also connected by various relationships…