2 citations · 2 across the 3 of their papers we have counts for
8 papers
TangoBERT: Reducing Inference Cost by using Cascaded Architecture
Jonathan Mamou, Oren Pereg, Moshe Wasserblat +1
The remarkable success of large transformer-based models such as BERT, RoBERTa and XLNet in many NLP tasks comes with a large increase in monetary and environmental cost due to the…
Expected Validation Performance and Estimation of a Random Variable's Maximum
Jesse Dodge, Suchin Gururangan, Dallas Card +2
Research in NLP is often supported by experimental results, and improved reporting of such results can lead to better understanding and more reproducible science. In this paper we…
Data Efficient Masked Language Modeling for Vision and Language
Yonatan Bitton, Gabriel Stanovsky, Michael Elhadad +1
Masked language modeling (MLM) is one of the key sub-tasks in vision-language pretraining. In the cross-modal setting, tokens in the sentence are masked at random, and the model pr…
RNN Architecture Learning with Sparse Regularization
Jesse Dodge, Roy Schwartz, Hao Peng +1
Neural models for NLP typically use large numbers of parameters to reach state-of-the-art performance, which can lead to excessive memory usage and increased runtime. We present a…
PaLM: A Hybrid Parser and Language Model
Hao Peng, Roy Schwartz, Noah A. Smith
We present PaLM, a hybrid parser and neural language model. Building on an RNN language model, PaLM adds an attention layer over text spans in the left context. An unsupervised con…
Green AI
Roy Schwartz, Jesse Dodge, Noah A. Smith +1
The computations required for deep learning research have been doubling every few months, resulting in an estimated 300,000x increase from 2012 to 2018 [2]. These computations have…