221 citations · 1.1k across the 24 of their papers we have counts for
11 papers · 1 filter
On the Expressivity Role of LayerNorm in Transformers' Attention
Shaked Brody, Uri Alon, Eran Yahav
Layer Normalization (LayerNorm) is an inherent component in all Transformer-based models. In this paper, we show that LayerNorm is crucial to the expressivity of the multi-head att…
Oversquashing in GNNs through the lens of information contraction and graph expansion
Pradeep Kr. Banerjee, Kedar Karhadkar, Yu Guang Wang +2
The quality of signal propagation in message-passing graph neural networks (GNNs) strongly influences their expressivity as has been observed in recent works. In particular, for pr…
How Attentive are Graph Attention Networks?
Shaked Brody, Uri Alon, Eran Yahav
Graph Attention Networks (GATs) are one of the most popular GNN architectures and are considered as the state-of-the-art architecture for representation learning with graphs. In GA…
Single-Node Attacks for Fooling Graph Neural Networks
Ben Finkelshtein, Chaim Baskin, Evgenii Zheltonozhskii +1
Graph neural networks (GNNs) have shown broad applicability in a variety of domains. These domains, e.g., social networks and product recommendations, are fertile ground for malici…
On the Bottleneck of Graph Neural Networks and its Practical Implications
Uri Alon, Eran Yahav
Since the proposal of the graph neural network (GNN) by Gori et al. (2005) and Scarselli et al. (2008), one of the major problems in training GNNs was their struggle to propagate i…
Adversarial Examples for Models of Code
Noam Yefet, Uri Alon, Eran Yahav
Neural models of code have shown impressive results when performing tasks such as predicting method names and identifying certain kinds of bugs. We show that these models are vulne…
Structural Language Models of Code
Uri Alon, Roy Sadaka, Omer Levy +1
We address the problem of any-code completion - generating a missing piece of source code in a given program without any restriction on the vocabulary or structure. We introduce a…
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Jonathan Shen, Patrick Nguyen, Yonghui Wu +88
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models a…
Neural Reverse Engineering of Stripped Binaries using Augmented Control Flow Graphs
Yaniv David, Uri Alon, Eran Yahav
We address the problem of reverse engineering of stripped executables, which contain no debug information. This is a challenging problem because of the low amount of syntactic info…
code2seq: Generating Sequences from Structured Representations of Code
Uri Alon, Shaked Brody, Omer Levy +1
The ability to generate natural language sequences from source code snippets has a variety of applications such as code summarization, documentation, and retrieval. Sequence-to-seq…
code2vec: Learning Distributed Representations of Code
Uri Alon, Meital Zilberstein, Omer Levy +1
We present a neural model for representing snippets of code as continuous distributed vectors ("code embeddings"). The main idea is to represent a code snippet as a single fixed-le…