644 citations · 782 across the 16 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.CL2022★ 1 cited
Simple Recurrence Improves Masked Language Models
Tao Lei, Ran Tian, Jasmijn Bastings +1
In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into t…
cs.CL2022★ 3 cited
Training Language Models with Memory Augmentation
Zexuan Zhong, Tao Lei, Danqi Chen
Recent work has improved language models (LMs) remarkably by equipping them with a non-parametric memory component. However, most existing approaches only introduce mem-ories at te…