16 citations · 33 across the 10 of their papers we have counts for
10 papers
Single-Nodal Spontaneous Symmetry Breaking in NLP Models
Shalom Rosner, Ronit D. Gross, Ella Koresh +1
Spontaneous symmetry breaking in statistical mechanics primarily occurs during phase transitions at the thermodynamic limit where the Hamiltonian preserves inversion symmetry, yet…
Self-attention vector output similarities reveal how machines pay attention
Tal Halevi, Yarden Tzach, Ronit D. Gross +2
The self-attention mechanism has significantly advanced the field of natural language processing, facilitating the development of advanced language-learning machines. Although its…
Translation Entropy: A Statistical Framework for Evaluating Translation Systems
Ronit D. Gross, Yanir Harel, Ido Kanter
The translation of written language has been known since the 3rd century BC; however, its necessity has become increasingly common in the information age. Today, many translators e…
Learning Mechanism Underlying NLP Pre-Training and Fine-Tuning
Yarden Tzach, Ronit D. Gross, Ella Koresh +4
Natural language processing (NLP) enables the understanding and generation of meaningful human language, typically using a pre-trained complex architecture on a large dataset to le…
Tiny language models
Ronit D. Gross, Yarden Tzach, Tal Halevi +2
A prominent achievement of natural language processing (NLP) is its ability to understand and generate meaningful human language. This capability relies on complex feedforward tran…
Low-latency vision transformers via large-scale multi-head attention
Ronit D. Gross, Tal Halevi, Ella Koresh +2
The emergence of spontaneous symmetry breaking among a few heads of multi-head attention (MHA) across transformer blocks in classification tasks was recently demonstrated through t…
Unified CNNs and transformers underlying learning mechanism reveals multi-head attention modus vivendi
Ella Koresh, Ronit D. Gross, Yuval Meir +3
Convolutional neural networks (CNNs) evaluate short-range correlations in input images which progress along the layers, whereas vision transformer (ViT) architectures evaluate long…
Advanced deep architecture pruning using single filter performance
Yarden Tzach, Yuval Meir, Ronit D. Gross +3
Pruning the parameters and structure of neural networks reduces the computational complexity, energy consumption, and latency during inference. Recently, a novel underlying mechani…
Role of Delay in Brain Dynamics
Yuval Meir, Ofek Tevet, Yarden Tzach +2
Significant variations of delays among connecting neurons cause an inevitable disadvantage of asynchronous brain dynamics compared to synchronous deep learning. However, this study…
Towards a universal mechanism for successful deep learning
Yuval Meir, Yarden Tzach, Shiri Hodassman +2
Recently, the underlying mechanism for successful deep learning (DL) was presented based on a quantitative method that measures the quality of a single filter in each layer of a DL…