A Pure Transformer Pretraining Framework on Text-attributed Graphs

Yu Song, Haitao Mao, Jiachen Xiao +6

Pretraining plays a pivotal role in acquiring generalized knowledge from large-scale data, achieving remarkable successes as evidenced by large models in CV and NLP. However, progr…