419 citations · 2.1k across the 46 of their papers we have counts for
3 papers · 1 filter
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design
Yoav Levine, Noam Wies, Daniel Jannai +3
Pretraining Neural Language Models (NLMs) over a large corpus involves chunking the text into training examples, which are contiguous text segments of sizes processable by the neur…
Which transformer architecture fits my data? A vocabulary bottleneck in self-attention
Noam Wies, Yoav Levine, Daniel Jannai +1
After their successful debut in natural language processing, Transformer architectures are now becoming the de-facto standard in many domains. An obstacle for their deployment over…
Neural tensor contractions and the expressive power of deep neural quantum states
Or Sharir, Amnon Shashua, Giuseppe Carleo
We establish a direct connection between general tensor networks and deep feed-forward artificial neural networks. The core of our results is the construction of neural-network lay…