3 papers
cs.LG2025
Attention Beyond Neighborhoods: Reviving Transformer for Graph Clustering
Xuanting Xie, Bingheng Li, Erlin Pan +3
Attention mechanisms have become a cornerstone in modern neural networks, driving breakthroughs across diverse domains. However, their application to graph structured data, where c…
cs.LG2025
Evaluating Loss Functions for Graph Neural Networks: Towards Pretraining and Generalization
Khushnood Abbas, Ruizhe Hou, Zhou Wengang +4
Graph Neural Networks (GNNs) became useful for learning on non-Euclidean data. However, their best performance depends on choosing the right model architecture and the training obj…
cs.CL2023
Co-training and Co-distillation for Quality Improvement and Compression of Language Models
Hayeon Lee, Rui Hou, Jongpil Kim +4
Knowledge Distillation (KD) compresses computationally expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models, allowing their use in resourc…