166 citations · 515 across the 60 of their papers we have counts for
67 papers
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Yi Pan, Haocheng Xi, Kan Zhu +10
Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Attention (KDA). Although they comp…
Complement Submodular Information Measures for Balanced and Robust Data Selection
Rishabh Iyer
Submodular optimization has become a fundamental paradigm for data selection, retrieval, summarization, and representation learning due to its ability to model coverage, diversity,…
SMA: Submodular Modality Aligner For Data Efficient Multimodal Learning
Truong Pham, Anay Majee, Rishabh Iyer
Despite the recent success of Multimodal Foundation Models (FMs), their reliance on massive paired datasets limits their applicability in low-data and rare-scenario settings where…
Gradient Coreset for Federated Learning
Durga Sivasubramanian, Lokesh Nagalapatti, Rishabh Iyer +1
Federated Learning (FL) is used to learn machine learning models with data that is partitioned across multiple clients, including resource-constrained edge devices. It is therefore…
SCoRe: Submodular Combinatorial Representation Learning
Anay Majee, Suraj Kothawade, Krishnateja Killamsetty +1
In this paper we introduce the SCoRe (Submodular Combinatorial Representation Learning) framework, a novel approach in representation learning that addresses inter-class bias and i…
Beyond Active Learning: Leveraging the Full Potential of Human Interaction via Auto-Labeling, Human Correction, and Human Verification
Nathan Beck, Krishnateja Killamsetty, Suraj Kothawade +1
Active Learning (AL) is a human-in-the-loop framework to interactively and adaptively label data instances, thereby enabling significant gains in model performance compared to rand…
STREAMLINE: Streaming Active Learning for Realistic Multi-Distributional Settings
Nathan Beck, Suraj Kothawade, Pradeep Shenoy +1
Deep neural networks have consistently shown great performance in several real-world use cases like autonomous vehicles, satellite imaging, etc., effectively leveraging large corpo…
INGENIOUS: Using Informative Data Subsets for Efficient Pre-Training of Language Models
H S V N S Kowndinya Renduchintala, Krishnateja Killamsetty, Sumit Bhatia +4
A salient characteristic of pre-trained language models (PTLMs) is a remarkable improvement in their generalization capability and emergence of new capabilities with increasing mod…
MILO: Model-Agnostic Subset Selection Framework for Efficient Model Training and Tuning
Krishnateja Killamsetty, Alexandre V. Evfimievski, Tejaswini Pedapati +3
Training deep networks and tuning hyperparameters on large datasets is computationally intensive. One of the primary research directions for efficient training is to reduce trainin…
Speeding up NAS with Adaptive Subset Selection
Vishak Prasad C, Colin White, Paarth Jain +2
A majority of recent developments in neural architecture search (NAS) have been aimed at decreasing the computational cost of various techniques without affecting their final perfo…
Partitioned Gradient Matching-based Data Subset Selection for Compute-Efficient Robust ASR Training
Ashish Mittal, Durga Sivasubramanian, Rishabh Iyer +2
Training state-of-the-art ASR systems such as RNN-T often has a high associated financial and environmental cost. Training with a subset of training data could mitigate this proble…
DIAGNOSE: Avoiding Out-of-distribution Data using Submodular Information Measures
Suraj Kothawade, Akshit Srivastava, Venkat Iyer +2
Avoiding out-of-distribution (OOD) data is critical for training supervised machine learning models in the medical imaging domain. Furthermore, obtaining labeled medical data is di…
CLINICAL: Targeted Active Learning for Imbalanced Medical Image Classification
Suraj Kothawade, Atharv Savarkar, Venkat Iyer +3
Training deep learning models on medical datasets that perform well for all classes is a challenging task. It is often the case that a suboptimal performance is obtained on some cl…
Active Data Discovery: Mining Unknown Data using Submodular Information Measures
Suraj Kothawade, Shivang Chopra, Saikat Ghosh +1
Active Learning is a very common yet powerful framework for iteratively and adaptively sampling subsets of the unlabeled sets with a human in the loop with the goal of achieving la…
AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning
Krishnateja Killamsetty, Guttu Sai Abhishek, Aakriti +4
Deep neural networks have seen great success in recent years; however, training a deep model is often challenging as its performance heavily depends on the hyper-parameters used. I…
BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets
Suraj Kothawade, Pavan Kumar Reddy, Ganesh Ramakrishnan +1
Current semi-supervised learning (SSL) methods assume a balance between the number of data points available for each class in both the labeled and the unlabeled data sets. However,…
Submodlib: A Submodular Optimization Library
Vishal Kaushal, Ganesh Ramakrishnan, Rishabh Iyer
Submodular functions are a special class of set functions which naturally model the notion of representativeness, diversity, coverage etc. and have been shown to be computationally…
PLATINUM: Semi-Supervised Model Agnostic Meta-Learning using Submodular Mutual Information
Changbin Li, Suraj Kothawade, Feng Chen +1
Few-shot classification (FSC) requires training models using a few (typically one to five) data points per class. Meta learning has proven to be able to learn a parametrized model…
TALISMAN: Targeted Active Learning for Object Detection with Rare Classes and Slices using Submodular Mutual Information
Suraj Kothawade, Saikat Ghosh, Sumit Shekhar +2
Deep neural networks based object detectors have shown great success in a variety of domains like autonomous vehicles, biomedical imaging, etc. It is known that their success depen…
GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning
Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer +1
Continual learning (CL) aims to develop techniques by which a single model adapts to an increasing number of tasks encountered sequentially, thereby potentially leveraging learning…
DITTO: Data-efficient and Fair Targeted Subset Selection for ASR Accent Adaptation
Suraj Kothawade, Anmol Mekala, Chandra Sekhara D +4
State-of-the-art Automatic Speech Recognition (ASR) systems are known to exhibit disparate performance on varying speech accents. To improve performance on a specific target accent…
Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming
Ayush Maheshwari, Krishnateja Killamsetty, Ganesh Ramakrishnan +3
A critical bottleneck in supervised machine learning is the need for large amounts of labeled data which is expensive and time consuming to obtain. However, it has been shown that…
Independence Properties of Generalized Submodular Information Measures
Himanshu Asnani, Jeff Bilmes, Rishabh Iyer
Recently a class of generalized information measures was defined on sets of items parametrized by submodular functions. In this paper, we propose and study various notions of indep…
SPEAR : Semi-supervised Data Programming in Python
Guttu Sai Abhishek, Harshad Ingole, Parth Laturia +4
We present SPEAR, an open-source python library for data programming with semi supervision. The package implements several recent data programming approaches including facility to…