57 citations · 64 across the 13 of their papers we have counts for
5 papers · 1 filter
Computationally-efficient Graph Modeling with Refined Graph Random Features
Krzysztof Choromanski, Avinava Dubey, Arijit Sehanobish +1
We propose refined GRFs (GRFs++), a new class of Graph Random Features (GRFs) for efficient and accurate computations involving kernels defined on the nodes of a graph. GRFs++ reso…
Learning the RoPEs: Better 2D and 3D Position Encodings with STRING
Connor Schenck, Isaac Reid, Mithun George Jacob +19
We introduce STRING: Separable Translationally Invariant Position Encodings. STRING extends Rotary Position Encodings, a recently proposed and widely used algorithm in large langua…
Linear Transformer Topological Masking with Graph Random Features
Isaac Reid, Kumar Avinava Dubey, Deepali Jain +12
When training transformers on graph-structured data, incorporating information about the underlying topology is crucial for good performance. Topological masking, a type of relativ…
Optimal Time Complexity Algorithms for Computing General Random Walk Graph Kernels on Sparse Graphs
Krzysztof Choromanski, Isaac Reid, Arijit Sehanobish +1
We present the first linear time complexity randomized algorithms for unbiased approximation of the celebrated family of general random walk kernels (RWKs) for sparse graphs. This…
Why flatness does and does not correlate with generalization for deep neural networks
Shuofeng Zhang, Isaac Reid, Guillermo Valle Pérez +1
The intuition that local flatness of the loss landscape is correlated with better generalization for deep neural networks (DNNs) has been explored for decades, spawning many differ…