281 citations · 873 across the 21 of their papers we have counts for
Showing 2022Show all
3 papers · 1 filter
cs.LG2022★ 37 cited
Mega: Moving Average Equipped Gated Attention
Xuezhe Ma, Chunting Zhou, Xiang Kong +5
The design choices in the Transformer attention mechanism, including weak inductive bias and quadratic computational complexity, have limited its application for modeling long sequ…
cs.CL2022★ 4 cited
Prompt Consistency for Zero-Shot Task Generalization
Chunting Zhou, Junxian He, Xuezhe Ma +2
One of the most impressive results of recent NLP history is the ability of pre-trained language models to solve new tasks in a zero-shot setting. To achieve this, NLP tasks are fra…
cs.CL2022★ 19 cited
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval
Uri Alon, Frank F. Xu, Junxian He +3
Retrieval-based language models (R-LM) model the probability of natural language text by combining a standard language model (LM) with examples retrieved from an external datastore…