12 papers
Computationally-efficient Graph Modeling with Refined Graph Random Features
Krzysztof Choromanski, Avinava Dubey, Arijit Sehanobish +1
We propose refined GRFs (GRFs++), a new class of Graph Random Features (GRFs) for efficient and accurate computations involving kernels defined on the nodes of a graph. GRFs++ reso…
Rotary Position Encodings for Graphs
Isaac Reid, Arijit Sehanobish, Cederik Höfs +7
We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transform…
SWING: Unlocking Implicit Graph Representations for Graph Random Features
Alessandro Manenti, Avinava Dubey, Arijit Sehanobish +2
We propose SWING: Space Walks for Implicit Network Graphs, a new class of algorithms for computations involving Graph Random Features on graphs given by implicit representations (i…
Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
Arijit Sehanobish, Charles Lovering
We study the \textit{parameter placement problem}: given a fixed budget of trainable entries within the B matrix of a LoRA adapter (A frozen), does the choice of which matt…
RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodings
Byeongchan Kim, Arijit Sehanobish, Avinava Dubey +2
We present a new class of efficient attention mechanisms applying universal 3D Relative Positional Encoding (RPE) methods given by arbitrary integrable modulation functions . Th…
EUGens: Efficient, Unified, and General Dense Layers
Sang Min Kim, Byeongchan Kim, Arijit Sehanobish +7
Efficient neural networks are essential for scaling machine learning models to real-time applications and resource-constrained environments. Fully-connected feedforward layers (FFL…