261 citations · 890 across the 15 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
cs.CL2020★ 1 cited
Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation
Yoel Zeldes, Dan Padnos, Or Sharir +1
We introduce a simple and efficient method, called Auxiliary Tuning, for adapting a pre-trained Language Model to a novel task; we demonstrate this approach on the task of conditio…
cs.LG2020★ 7 cited
The Depth-to-Width Interplay in Self-Attention
Yoav Levine, Noam Wies, Or Sharir +2
Self-attention architectures, which are rapidly pushing the frontier in natural language processing, demonstrate a surprising depth-inefficient behavior: previous works indicate th…
cs.CL2020★ 114 cited
The Cost of Training NLP Models: A Concise Overview
Or Sharir, Barak Peleg, Yoav Shoham
We review the cost of training large-scale language models, and the drivers of these costs. The intended audience includes engineers and scientists budgeting their model-training e…