From the 1 of 140 linked papers with an AI index.
3.7k citations · 25.2k across the 137 of their papers we have counts for
8 papers · 1 filter
Augmented Language Models: a Survey
Grégoire Mialon, Roberto Dessì, Maria Lomeli +10
This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. The former is defined as decomposing a potentially comple…
Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors
Zeyu Yun, Yubei Chen, Bruno A Olshausen +1
Transformer networks have revolutionized NLP representation learning since they were introduced. Though a great effort has been made to explain the representation in transformers,…
Adversarially-Trained Normalized Noisy-Feature Auto-Encoder for Text Generation
Xiang Zhang, Yann LeCun
This article proposes Adversarially-Trained Normalized Noisy-Feature Auto-Encoder (ATNNFAE) for byte-level text generation. An ATNNFAE consists of an auto-encoder where the interna…
Byte-Level Recursive Convolutional Auto-Encoder for Text
Xiang Zhang, Yann LeCun
This article proposes to auto-encode text at byte-level using convolutional networks with a recursive architecture. The motivation is to explore whether it is possible to have scal…
Which Encoding is the Best for Text Classification in Chinese, English, Japanese and Korean?
Xiang Zhang, Yann LeCun
This article offers an empirical study on the different ways of encoding Chinese, Japanese, Korean (CJK) and English languages for text classification. Different encoding levels ar…
Tracking the World State with Recurrent Entity Networks
Mikael Henaff, Jason Weston, Arthur Szlam +2
We introduce a new model, the Recurrent Entity Network (EntNet). It is equipped with a dynamic long-term memory which allows it to maintain and update a representation of the state…