89 citations · 219 across the 11 of their papers we have counts for
13 papers
BARTSmiles: Generative Masked Language Models for Molecular Representations
Gayane Chilingaryan, Hovhannes Tamoyan, Ani Tevosyan +6
We discover a robust self-supervised strategy tailored towards molecular representations for generative masked language models through a series of tailored, in-depth ablations. Usi…
CM3: A Causal Masked Multimodal Model of the Internet
Armen Aghajanyan, Bernie Huang, Candace Ross +8
We introduce CM3, a family of causally masked generative models trained over a large corpus of structured multi-modal documents that can contain both text and image tokens. Our new…
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
Hu Xu, Gargi Ghosh, Po-Yao Huang +5
We present VideoCLIP, a contrastive approach to pre-train a unified model for zero-shot video and text understanding, without using any labels on downstream tasks. VideoCLIP trains…
RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing
Vivek Gupta, Akshat Shrivastava, Adithya Sagar +2
While large pre-trained language models accumulate a lot of knowledge in their parameters, it has been demonstrated that augmenting it with non-parametric retrieval-based memory ha…
HTLM: Hyper-Text Pre-Training and Prompting of Language Models
Armen Aghajanyan, Dmytro Okhonko, Mike Lewis +4
We introduce HTLM, a hyper-text language model trained on a large-scale web crawl. Modeling hyper-text has a number of advantages: (1) it is easily gathered at scale, (2) it provid…
Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog
Arun Babu, Akshat Shrivastava, Armen Aghajanyan +3
Semantic parsing using sequence-to-sequence models allows parsing of deeper representations compared to traditional word tagging based models. In spite of these advantages, widespr…