273 citations · 751 across the 44 of their papers we have counts for
6 papers · 2 filters
Linear Transformer Topological Masking with Graph Random Features
Isaac Reid, Kumar Avinava Dubey, Deepali Jain +12
When training transformers on graph-structured data, incorporating information about the underlying topology is crucial for good performance. Topological masking, a type of relativ…
Optimal Time Complexity Algorithms for Computing General Random Walk Graph Kernels on Sparse Graphs
Krzysztof Choromanski, Isaac Reid, Arijit Sehanobish +1
We present the first linear time complexity randomized algorithms for unbiased approximation of the celebrated family of general random walk kernels (RWKs) for sparse graphs. This…
Conditional Language Policy: A General Framework for Steerable Multi-Objective Finetuning
Kaiwen Wang, Rahul Kidambi, Ryan Sullivan +17
Reward-based finetuning is crucial for aligning language policies with intended behaviors (e.g., creativity and safety). A key challenge is to develop steerable language models tha…
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning
Arijit Sehanobish, Avinava Dubey, Krzysztof Choromanski +4
Recent efforts to scale Transformer models have demonstrated rapid progress across a wide range of tasks (Wei et al., 2022). However, fine-tuning these models for downstream tasks…
Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning
Somnath Basu Roy Chowdhury, Krzysztof Choromanski, Arijit Sehanobish +2
Machine unlearning is the process of efficiently removing the influence of a training data instance from a trained machine learning model without retraining it from scratch. A popu…
Fast Tree-Field Integrators: From Low Displacement Rank to Topological Transformers
Krzysztof Choromanski, Arijit Sehanobish, Somnath Basu Roy Chowdhury +4
We present a new class of fast polylog-linear algorithms based on the theory of structured matrices (in particular low displacement rank) for integrating tensor fields defined on w…