208 citations · 370 across the 13 of their papers we have counts for
9 papers · 1 filter
IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models
Chenguang Wang, Xiao Liu, Dawn Song
We introduce a new open information extraction (OIE) benchmark for pre-trained language models (LM). Recent studies have demonstrated that pre-trained LMs, such as BERT and GPT, ma…
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion
Jianhao Shen, Chenguang Wang, Ye Yuan +5
This paper presents a parameter-lite transfer learning approach of pretrained language models (LM) for knowledge graph (KG) completion. Instead of finetuning, which modifies all LM…
Joint Language Semantic and Structure Embedding for Knowledge Graph Completion
Jianhao Shen, Chenguang Wang, Linyuan Gong +1
The task of completing knowledge triplets has broad downstream applications. Both structural and semantic information plays an important role in knowledge graph completion. Unlike…
DeepStruct: Pretraining of Language Models for Structure Prediction
Chenguang Wang, Xiao Liu, Zui Chen +3
We introduce a method for improving the structural understanding abilities of language models. Unlike previous approaches that finetune the models with task-specific augmentation,…
Zero-Shot Information Extraction as a Unified Text-to-Triple Translation
Chenguang Wang, Xiao Liu, Zui Chen +3
We cast a suite of information extraction tasks into a text-to-triple translation framework. Instead of solving each task relying on task-specific datasets and models, we formalize…
Language Models are Open Knowledge Graphs
Chenguang Wang, Xiao Liu, Dawn Song
This paper shows how to construct knowledge graphs (KGs) from pre-trained language models (e.g., BERT, GPT-2/3), without human supervision. Popular KGs (e.g, Wikidata, NELL) are bu…
Transformer on a Diet
Chenguang Wang, Zihao Ye, Aston Zhang +2
Transformer has been widely used thanks to its ability to capture sequence information in an efficient way. However, recent developments, such as BERT and GPT-2, deliver only heavy…
PoD: Positional Dependency-Based Word Embedding for Aspect Term Extraction
Yichun Yin, Chenguang Wang, Ming Zhang
Dependency context-based word embedding jointly learns the representations of word and dependency context, and has been proved effective in aspect term extraction. In this paper, w…
Language Models with Transformers
Chenguang Wang, Mu Li, Alexander J. Smola
The Transformer architecture is superior to RNN-based models in computational efficiency. Recently, GPT and BERT demonstrate the efficacy of Transformer models on various NLP tasks…