3 papers
cs.CL2022
Relational Memory Augmented Language Models
Qi Liu, Dani Yogatama, Phil Blunsom
We present a memory-augmented approach to condition an autoregressive language model on a knowledge graph. We represent the graph as a collection of relation triples and retrieve r…
cs.CL2021
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application
Chuhan Wu, Fangzhao Wu, Yang Yu +3
Pre-trained language models (PLMs) like BERT have made great progress in NLP. News articles usually contain rich textual information, and PLMs have the potentials to enhance news t…
cs.CL2020
A Survey on Contextual Embeddings
Qi Liu, Matt J. Kusner, Phil Blunsom
Contextual embeddings, such as ELMo and BERT, move beyond global word representations like Word2Vec and achieve ground-breaking performance on a wide range of natural language proc…