6 papers
Graph Random Features for Scalable Gaussian Processes
Matthew Zhang, Jihao Andreas Lin, Krzysztof Choromanski +3
We study the application of graph random features (GRFs) - a recently introduced stochastic estimator of graph node kernels - to scalable Gaussian processes on discrete input space…
Rotary Position Encodings for Graphs
Isaac Reid, Arijit Sehanobish, Cederik Höfs +7
We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transform…
Linear Transformer Topological Masking with Graph Random Features
Isaac Reid, Kumar Avinava Dubey, Deepali Jain +12
When training transformers on graph-structured data, incorporating information about the underlying topology is crucial for good performance. Topological masking, a type of relativ…
Variance-Reducing Couplings for Random Features
Isaac Reid, Stratis Markou, Krzysztof Choromanski +2
Random features (RFs) are a popular technique to scale up kernel methods in machine learning, replacing exact kernel evaluations with stochastic Monte Carlo estimates. They underpi…
General Graph Random Features
Isaac Reid, Krzysztof Choromanski, Eli Berger +1
We propose a novel random walk-based algorithm for unbiased estimation of arbitrary functions of a weighted adjacency matrix, coined universal graph random features (u-GRFs). This…
Repelling Random Walks
Isaac Reid, Eli Berger, Krzysztof Choromanski +1
We present a novel quasi-Monte Carlo mechanism to improve graph-based sampling, coined repelling random walks. By inducing correlations between the trajectories of an interacting e…