273 citations · 751 across the 44 of their papers we have counts for
7 papers · 2 filters
Robust Concept Erasure via Kernelized Rate-Distortion Maximization
Somnath Basu Roy Chowdhury, Nicholas Monath, Avinava Dubey +2
Distributed representations provide a vector space that captures meaningful relationships between data instances. The distributed nature of these representations, however, entangle…
Scalable Neural Network Kernels
Arijit Sehanobish, Krzysztof Choromanski, Yunfan Zhao +2
We introduce the concept of scalable neural network kernels (SNNKs), the replacements of regular feedforward layers (FFLs), capable of approximating the latter, but with favorable…
Enhancing Group Fairness in Online Settings Using Oblique Decision Forests
Somnath Basu Roy Chowdhury, Nicholas Monath, Ahmad Beirami +4
Fairness, especially group fairness, is an important consideration in the context of machine learning systems. The most commonly adopted group fairness-enhancing techniques are in-…
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
Krzysztof Marcin Choromanski, Shanda Li, Valerii Likhosherstov +7
We propose a new class of linear Transformers called FourierLearner-Transformers (FLTs), which incorporate a wide range of relative positional encoding mechanisms (RPEs). These inc…
Mnemosyne: Learning to Train Transformers with Transformers
Deepali Jain, Krzysztof Marcin Choromanski, Avinava Dubey +4
In this work, we propose a new class of learnable optimizers, called \textit{Mnemosyne}. It is based on the novel spatio-temporal low-rank implicit attention Transformers that can…
Efficient Graph Field Integrators Meet Point Clouds
Krzysztof Choromanski, Arijit Sehanobish, Han Lin +13
We present two new classes of algorithms for efficient field integration on graphs encoding point clouds. The first class, SeparatorFactorization(SF), leverages the bounded genus o…