7 citations · 34 across the 25 of their papers we have counts for
20 papers · 1 filter
Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
Arijit Sehanobish, Charles Lovering
We study the \textit{parameter placement problem}: given a fixed budget of trainable entries within the B matrix of a LoRA adapter (A frozen), does the choice of which matt…
RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodings
Byeongchan Kim, Arijit Sehanobish, Avinava Dubey +2
We present a new class of efficient attention mechanisms applying universal 3D Relative Positional Encoding (RPE) methods given by arbitrary integrable modulation functions . Th…
EUGens: Efficient, Unified, and General Dense Layers
Sang Min Kim, Byeongchan Kim, Arijit Sehanobish +7
Efficient neural networks are essential for scaling machine learning models to real-time applications and resource-constrained environments. Fully-connected feedforward layers (FFL…
SWING: Unlocking Implicit Graph Representations for Graph Random Features
Alessandro Manenti, Avinava Dubey, Arijit Sehanobish +2
We propose SWING: Space Walks for Implicit Network Graphs, a new class of algorithms for computations involving Graph Random Features on graphs given by implicit representations (i…
Computationally-efficient Graph Modeling with Refined Graph Random Features
Krzysztof Choromanski, Avinava Dubey, Arijit Sehanobish +1
We propose refined GRFs (GRFs++), a new class of Graph Random Features (GRFs) for efficient and accurate computations involving kernels defined on the nodes of a graph. GRFs++ reso…
Rotary Position Encodings for Graphs
Isaac Reid, Arijit Sehanobish, Cederik Höfs +7
We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transform…
Optimal Time Complexity Algorithms for Computing General Random Walk Graph Kernels on Sparse Graphs
Krzysztof Choromanski, Isaac Reid, Arijit Sehanobish +1
We present the first linear time complexity randomized algorithms for unbiased approximation of the celebrated family of general random walk kernels (RWKs) for sparse graphs. This…
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning
Arijit Sehanobish, Avinava Dubey, Krzysztof Choromanski +4
Recent efforts to scale Transformer models have demonstrated rapid progress across a wide range of tasks (Wei et al., 2022). However, fine-tuning these models for downstream tasks…
Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning
Somnath Basu Roy Chowdhury, Krzysztof Choromanski, Arijit Sehanobish +2
Machine unlearning is the process of efficiently removing the influence of a training data instance from a trained machine learning model without retraining it from scratch. A popu…
Fast Tree-Field Integrators: From Low Displacement Rank to Topological Transformers
Krzysztof Choromanski, Arijit Sehanobish, Somnath Basu Roy Chowdhury +4
We present a new class of fast polylog-linear algorithms based on the theory of structured matrices (in particular low displacement rank) for integrating tensor fields defined on w…
Scalable Neural Network Kernels
Arijit Sehanobish, Krzysztof Choromanski, Yunfan Zhao +2
We introduce the concept of scalable neural network kernels (SNNKs), the replacements of regular feedforward layers (FFLs), capable of approximating the latter, but with favorable…
Efficient Graph Field Integrators Meet Point Clouds
Krzysztof Choromanski, Arijit Sehanobish, Han Lin +13
We present two new classes of algorithms for efficient field integration on graphs encoding point clouds. The first class, SeparatorFactorization(SF), leverages the bounded genus o…
Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks
Arijit Sehanobish, Kawshik Kannan, Nabila Abraham +2
Large pretrained Transformer-based language models like BERT and GPT have changed the landscape of Natural Language Processing (NLP). However, fine tuning such models still require…
Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports
Arijit Sehanobish, McCullen Sandora, Nabila Abraham +7
Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP. However, training or fine-tuning these models for individual tasks can b…
Efficient Extraction of Pathologies from C-Spine Radiology Reports using Multi-Task Learning
Arijit Sehanobish, Nathaniel Brown, Ishita Daga +7
Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP. Generally, if one has multiple tasks on a given dataset, one may finetun…
Hybrid Random Features
Krzysztof Choromanski, Haoxian Chen, Han Lin +10
We propose a new class of random feature methods for linearizing softmax and Gaussian kernels called hybrid random features (HRFs) that automatically adapt the quality of kernel es…
From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers
Krzysztof Choromanski, Han Lin, Haoxian Chen +7
In this paper we provide, to the best of our knowledge, the first comprehensive approach for incorporating various masking mechanisms into Transformers architectures in a scalable…
Permutation invariant networks to learn Wasserstein metrics
Arijit Sehanobish, Neal Ravindra, David van Dijk
Understanding the space of probability measures on a metric space equipped with a Wasserstein distance is one of the fundamental questions in mathematical analysis. The Wasserstein…
Learning Potentials of Quantum Systems using Deep Neural Networks
Arijit Sehanobish, Hector H. Corzo, Onur Kara +1
Attempts to apply Neural Networks (NN) to a wide range of research problems have been ubiquitous and plentiful in recent literature. Particularly, the use of deep NNs for understan…
Gaining Insight into SARS-CoV-2 Infection and COVID-19 Severity Using Self-supervised Edge Features and Graph Neural Networks
Arijit Sehanobish, Neal G. Ravindra, David van Dijk
A molecular and cellular understanding of how SARS-CoV-2 variably infects and causes severe COVID-19 remains a bottleneck in developing interventions to end the pandemic. We sought…