37 citations · 82 across the 12 of their papers we have counts for
5 papers · 1 filter
Low-latency vision transformers via large-scale multi-head attention
Ronit D. Gross, Tal Halevi, Ella Koresh +2
The emergence of spontaneous symmetry breaking among a few heads of multi-head attention (MHA) across transformer blocks in classification tasks was recently demonstrated through t…
Towards a universal mechanism for successful deep learning
Yuval Meir, Yarden Tzach, Shiri Hodassman +2
Recently, the underlying mechanism for successful deep learning (DL) was presented based on a quantitative method that measures the quality of a single filter in each layer of a DL…
The mechanism underlying successful deep learning
Yarden Tzach, Yuval Meir, Ofek Tevet +4
Deep architectures consist of tens or hundreds of convolutional layers (CLs) that terminate with a few fully connected (FC) layers and an output layer representing the possible lab…
Enhancing the accuracies by performing pooling decisions adjacent to the output layer
Yuval Meir, Yarden Tzach, Ronit D. Gross +3
Learning classification tasks of (2^nx2^n) inputs typically consist of \le n (2x2) max-pooling (MP) operators along the entire feedforward deep architecture. Here we show, using th…
Learning on tree architectures outperforms a convolutional feedforward network
Yuval Meir, Itamar Ben-Noam, Yarden Tzach +2
Advanced deep learning architectures consist of tens of fully connected and convolutional hidden layers, currently extended to hundreds, are far from their biological realization.…