298 citations · 716 across the 23 of their papers we have counts for
23 papers
GraphAlign: Pretraining One Graph Neural Network on Multiple Graphs via Feature Alignment
Zhenyu Hou, Haozhan Li, Yukuo Cen +2
Graph self-supervised learning (SSL) holds considerable promise for mining and learning with graph-structured data. Yet, a significant challenge in graph SSL lies in the feature di…
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
Bo Chen, Xingyi Cheng, Pan Li +12
Protein language models have shown remarkable success in learning biological information from protein sequences. However, most existing models are limited by either autoencoding or…
AgentBench: Evaluating LLMs as Agents
Xiao Liu, Hao Yu, Hanchen Zhang +19
The potential of Large Language Model (LLM) as agents has been widely acknowledged recently. Thus, there is an urgent need to quantitatively \textit{evaluate LLMs as agents} on cha…
WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences
Xiao Liu, Hanyu Lai, Hao Yu +6
We present WebGLM, a web-enhanced question-answering system based on the General Language Model (GLM). Its goal is to augment a pre-trained large language model (LLM) with web sear…
BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphs
Zhen Yang, Tinglin Huang, Ming Ding +5
In-Batch contrastive learning is a state-of-the-art self-supervised method that brings semantically-similar instances close while pushing dissimilar instances apart within a mini-b…
Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deterioration
Kejuan Yang, Xiao Liu, Kaiwen Men +3
We identify two crucial limitations in the evaluation of recent parallel-integrated method Parallel Context Windows (PCW), which extends the maximum context lengths of language mod…
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner
Zhenyu Hou, Yufei He, Yukuo Cen +4
Graph self-supervised learning (SSL), including contrastive and generative approaches, offers great potential to address the fundamental challenge of label scarcity in real-world g…
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
Jiazheng Xu, Xiao Liu, Yuchen Wu +5
We present a comprehensive solution to learn and improve text-to-image models from human preference feedback. To begin with, we build ImageReward -- the first general-purpose text-…
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
Qinkai Zheng, Xiao Xia, Xu Zou +10
Large pre-trained code generation models, such as OpenAI Codex, can generate syntax- and function-correct code, making the coding of programmers more productive and our pursuit of…
GLM-130B: An Open Bilingual Pre-trained Model
Aohan Zeng, Xiao Liu, Zhengxiao Du +15
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good…
Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries
Xiao Liu, Shiyu Zhao, Kai Su +6
Knowledge graph (KG) embeddings have been a mainstream approach for reasoning over incomplete KGs. However, limited by their inherently shallow and static architectures, they can h…
Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers
Weng Lam Tam, Xiao Liu, Kaixuan Ji +6
Prompt tuning attempts to update few task-specific parameters in pre-trained models. It has achieved comparable performance to fine-tuning of the full parameter set on both languag…
Masked Autoencoders for Generic Event Boundary Detection CVPR'2022 Kinetics-GEBD Challenge
Rui He, Yuanxi Sun, Youzeng Li +4
Generic Event Boundary Detection (GEBD) tasks aim at detecting generic, taxonomy-free event boundaries that segment a whole video into chunks. In this paper, we apply Masked Autoen…
GACT: Activation Compressed Training for Generic Network Architectures
Xiaoxuan Liu, Lianmin Zheng, Dequan Wang +9
Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint.…
Rethinking the Setting of Semi-supervised Learning on Graphs
Ziang Li, Ming Ding, Weikai Li +4
We argue that the present setting of semisupervised learning on graphs may result in unfair comparisons, due to its potential risk of over-tuning hyper-parameters for models. In th…
GraphMAE: Self-Supervised Masked Graph Autoencoders
Zhenyu Hou, Xiao Liu, Yukuo Cen +4
Self-supervised learning (SSL) has been extensively explored in recent years. Particularly, generative SSL has seen emerging success in natural language processing and other AI fie…
GRAND+: Scalable Graph Random Neural Networks
Wenzheng Feng, Yuxiao Dong, Tinglin Huang +4
Graph neural networks (GNNs) have been widely adopted for semi-supervised learning on graphs. A recent study shows that the graph random neural network (GRAND) model can generate s…
SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs
Xiao Liu, Haoyun Hong, Xinghao Wang +4
Entity alignment, aiming to identify equivalent entities across different knowledge graphs (KGs), is a fundamental problem for constructing Web-scale KGs. Over the course of its de…
Training Free Graph Neural Networks for Graph Matching
Zhiyuan Liu, Yixin Cao, Fuli Feng +4
We present a framework of Training Free Graph Matching (TFGM) to boost the performance of Graph Neural Networks (GNNs) based graph matching, providing a fast promising solution wit…
SCR: Training Graph Neural Networks with Consistency Regularization
Chenhui Zhang, Yufei He, Yukuo Cen +7
We present the SCR framework for enhancing the training of graph neural networks (GNNs) with consistency regularization. Regularization is a set of strategies used in Machine Learn…
Graph Contrastive Learning for Anomaly Detection
Bo Chen, Jing Zhang, Xiaokang Zhang +6
Graph-based anomaly detection has been widely used for detecting malicious activities in real-world applications. Existing attempts to address this problem have thus far focused on…
OAG-BERT: Towards A Unified Backbone Language Model For Academic Knowledge Services
Xiao Liu, Da Yin, Jingnan Zheng +5
Academic knowledge services have substantially facilitated the development of the science enterprise by providing a plenitude of efficient research tools. However, many application…
CogDL: A Comprehensive Library for Graph Deep Learning
Yukuo Cen, Zhenyu Hou, Yan Wang +15
Graph neural networks (GNNs) have attracted tremendous attention from the graph learning community in recent years. It has been widely adopted in various real-world applications fr…