22 citations · 37 across the 6 of their papers we have counts for
7 papers
LaMemo: Language Modeling with Look-Ahead Memory
Haozhe Ji, Rongsheng Zhang, Zhenyu Yang +2
Although Transformers with fully connected self-attentions are powerful to model long-term dependencies, they are struggling to scale to long texts with thousands of words in langu…
DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer
Haozhe Ji, Minlie Huang
Despite the recent advances in applying pre-trained language models to generate high-quality texts, generating long passages that maintain long-range coherence is yet challenging f…
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs
Pei Ke, Haozhe Ji, Yu Ran +5
Existing pre-trained models for knowledge-graph-to-text (KG-to-text) generation simply fine-tune text-to-text pre-trained models such as BART or T5 on KG-to-text datasets, which la…
CPM: A Large-scale Generative Chinese Pre-trained Language Model
Zhengyan Zhang, Xu Han, Hao Zhou +22
Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570GB training data, drew a lot o…
Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Haozhe Ji, Pei Ke, Shaohan Huang +2
Commonsense explanation generation aims to empower the machine's sense-making capability by generating plausible explanations to statements against commonsense. While this task is…
Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Haozhe Ji, Pei Ke, Shaohan Huang +3
Despite the success of generative pre-trained language models on a series of text generation tasks, they still suffer in cases where reasoning over underlying commonsense knowledge…