787 citations · 4.6k across the 56 of their papers we have counts for
56 papers
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
Bo Chen, Xingyi Cheng, Pan Li +12
Protein language models have shown remarkable success in learning biological information from protein sequences. However, most existing models are limited by either autoencoding or…
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
Yushi Bai, Xin Lv, Jiajie Zhang +10
Although large language models (LLMs) demonstrate impressive performance for many language tasks, most of them can only handle texts a few thousand tokens long, limiting their appl…
AgentBench: Evaluating LLMs as Agents
Xiao Liu, Hao Yu, Hanchen Zhang +19
The potential of Large Language Model (LLM) as agents has been widely acknowledged recently. Thus, there is an urgent need to quantitatively \textit{evaluate LLMs as agents} on cha…
WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences
Xiao Liu, Hanyu Lai, Hao Yu +6
We present WebGLM, a web-enhanced question-answering system based on the General Language Model (GLM). Its goal is to augment a pre-trained large language model (LLM) with web sear…
BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphs
Zhen Yang, Tinglin Huang, Ming Ding +5
In-Batch contrastive learning is a state-of-the-art self-supervised method that brings semantically-similar instances close while pushing dissimilar instances apart within a mini-b…
Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deterioration
Kejuan Yang, Xiao Liu, Kaiwen Men +3
We identify two crucial limitations in the evaluation of recent parallel-integrated method Parallel Context Windows (PCW), which extends the maximum context lengths of language mod…
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner
Zhenyu Hou, Yufei He, Yukuo Cen +4
Graph self-supervised learning (SSL), including contrastive and generative approaches, offers great potential to address the fundamental challenge of label scarcity in real-world g…
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
Jiazheng Xu, Xiao Liu, Yuchen Wu +5
We present a comprehensive solution to learn and improve text-to-image models from human preference feedback. To begin with, we build ImageReward -- the first general-purpose text-…
CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X
Qinkai Zheng, Xiao Xia, Xu Zou +10
Large pre-trained code generation models, such as OpenAI Codex, can generate syntax- and function-correct code, making the coding of programmers more productive and our pursuit of…
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation
Ji Qi, Jifan Yu, Teng Tu +12
Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i.e., long and informative commen…
Web-Scale Academic Name Disambiguation: the WhoIsWho Benchmark, Leaderboard, and Toolkit
Bo Chen, Jing Zhang, Fanjin Zhang +5
Name disambiguation -- a fundamental problem in online academic systems -- is now facing greater challenges with the increasing growth of research papers. For example, on AMiner, a…
GLM-130B: An Open Bilingual Pre-trained Model
Aohan Zeng, Xiao Liu, Zhengxiao Du +15
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model at least as good…
Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries
Xiao Liu, Shiyu Zhao, Kai Su +6
Knowledge graph (KG) embeddings have been a mainstream approach for reasoning over incomplete KGs. However, limited by their inherently shallow and static architectures, they can h…
Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers
Weng Lam Tam, Xiao Liu, Kaixuan Ji +6
Prompt tuning attempts to update few task-specific parameters in pre-trained models. It has achieved comparable performance to fine-tuning of the full parameter set on both languag…
GraphMAE: Self-Supervised Masked Graph Autoencoders
Zhenyu Hou, Xiao Liu, Yukuo Cen +4
Self-supervised learning (SSL) has been extensively explored in recent years. Particularly, generative SSL has seen emerging success in natural language processing and other AI fie…
IDPG: An Instance-Dependent Prompt Generation Method
Zhuofeng Wu, Sinong Wang, Jiatao Gu +4
Prompt tuning is a new, efficient NLP transfer learning paradigm that adds a task-specific prompt in each input instance during the model training stage. It freezes the pre-trained…
GRAND+: Scalable Graph Random Neural Networks
Wenzheng Feng, Yuxiao Dong, Tinglin Huang +4
Graph neural networks (GNNs) have been widely adopted for semi-supervised learning on graphs. A recent study shows that the graph random neural network (GRAND) model can generate s…
SelfKG: Self-Supervised Entity Alignment in Knowledge Graphs
Xiao Liu, Haoyun Hong, Xinghao Wang +4
Entity alignment, aiming to identify equivalent entities across different knowledge graphs (KGs), is a fundamental problem for constructing Web-scale KGs. Over the course of its de…
EvoKG: Jointly Modeling Event Time and Network Structure for Reasoning over Temporal Knowledge Graphs
Namyong Park, Fuchen Liu, Purvanshi Mehta +3
How can we perform knowledge reasoning over temporal knowledge graphs (TKGs)? TKGs represent facts about entities and their relations, where each fact is associated with a timestam…
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Qingsong Lv, Ming Ding, Qiang Liu +7
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understandi…
SCR: Training Graph Neural Networks with Consistency Regularization
Chenhui Zhang, Yufei He, Yukuo Cen +7
We present the SCR framework for enhancing the training of graph neural networks (GNNs) with consistency regularization. Regularization is a set of strategies used in Machine Learn…
Graph Robustness Benchmark: Benchmarking the Adversarial Robustness of Graph Machine Learning
Qinkai Zheng, Xu Zou, Yuxiao Dong +5
Adversarial attacks on graphs have posed a major threat to the robustness of graph machine learning (GML) models. Naturally, there is an ever-escalating arms race between attackers…
Graph Contrastive Learning for Anomaly Detection
Bo Chen, Jing Zhang, Xiaokang Zhang +6
Graph-based anomaly detection has been widely used for detecting malicious activities in real-world applications. Existing attempts to address this problem have thus far focused on…
A Self-supervised Method for Entity Alignment
Xiao Liu, Haoyun Hong, Xinghao Wang +4
Entity alignment, aiming to identify equivalent entities across different knowledge graphs (KGs), is a fundamental problem for constructing large-scale KGs. Over the course of its…