1.1k citations · 2.4k across the 37 of their papers we have counts for
8 papers · 1 filter
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
Xiao Liu, Kaixuan Ji, Yicheng Fu +4
Prompt tuning, which only tunes continuous prompts with a frozen language model, substantially reduces per-task storage and memory usage at training. However, in the context of NLU…
Zero-Shot Information Extraction as a Unified Text-to-Triple Translation
Chenguang Wang, Xiao Liu, Zui Chen +3
We cast a suite of information extraction tasks into a text-to-triple translation framework. Instead of solving each task relying on task-specific datasets and models, we formalize…
A Self-supervised Method for Entity Alignment
Xiao Liu, Haoyun Hong, Xinghao Wang +4
Entity alignment, aiming to identify equivalent entities across different knowledge graphs (KGs), is a fundamental problem for constructing large-scale KGs. Over the course of its…
Pre-Trained Models: Past, Present and Future
Xu Han, Zhengyan Zhang, Ning Ding +21
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophis…
Understanding WeChat User Preferences and "Wow" Diffusion
Fanjin Zhang, Jie Tang, Xueyi Liu +9
WeChat is the largest social instant messaging platform in China, with 1.1 billion monthly active users. "Top Stories" is a novel friend-enhanced recommendation engine in WeChat, i…
GPT Understands, Too
Xiao Liu, Yanan Zheng, Zhengxiao Du +4
Prompting a pretrained language model with natural language patterns has been proved effective for natural language understanding (NLU). However, our preliminary study reveals that…
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
Zhengxiao Du, Yujie Qian, Xiao Liu +4
There have been various types of pretraining architectures including autoencoding models (e.g., BERT), autoregressive models (e.g., GPT), and encoder-decoder models (e.g., T5). How…
OAG-BERT: Towards A Unified Backbone Language Model For Academic Knowledge Services
Xiao Liu, Da Yin, Jingnan Zheng +5
Academic knowledge services have substantially facilitated the development of the science enterprise by providing a plenitude of efficient research tools. However, many application…