166 citations · 515 across the 60 of their papers we have counts for
40 papers · 1 filter
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Yi Pan, Haocheng Xi, Kan Zhu +10
Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Attention (KDA). Although they comp…
Complement Submodular Information Measures for Balanced and Robust Data Selection
Rishabh Iyer
Submodular optimization has become a fundamental paradigm for data selection, retrieval, summarization, and representation learning due to its ability to model coverage, diversity,…
SMA: Submodular Modality Aligner For Data Efficient Multimodal Learning
Truong Pham, Anay Majee, Rishabh Iyer
Despite the recent success of Multimodal Foundation Models (FMs), their reliance on massive paired datasets limits their applicability in low-data and rare-scenario settings where…
Gradient Coreset for Federated Learning
Durga Sivasubramanian, Lokesh Nagalapatti, Rishabh Iyer +1
Federated Learning (FL) is used to learn machine learning models with data that is partitioned across multiple clients, including resource-constrained edge devices. It is therefore…
SCoRe: Submodular Combinatorial Representation Learning
Anay Majee, Suraj Kothawade, Krishnateja Killamsetty +1
In this paper we introduce the SCoRe (Submodular Combinatorial Representation Learning) framework, a novel approach in representation learning that addresses inter-class bias and i…
Beyond Active Learning: Leveraging the Full Potential of Human Interaction via Auto-Labeling, Human Correction, and Human Verification
Nathan Beck, Krishnateja Killamsetty, Suraj Kothawade +1
Active Learning (AL) is a human-in-the-loop framework to interactively and adaptively label data instances, thereby enabling significant gains in model performance compared to rand…
STREAMLINE: Streaming Active Learning for Realistic Multi-Distributional Settings
Nathan Beck, Suraj Kothawade, Pradeep Shenoy +1
Deep neural networks have consistently shown great performance in several real-world use cases like autonomous vehicles, satellite imaging, etc., effectively leveraging large corpo…
MILO: Model-Agnostic Subset Selection Framework for Efficient Model Training and Tuning
Krishnateja Killamsetty, Alexandre V. Evfimievski, Tejaswini Pedapati +3
Training deep networks and tuning hyperparameters on large datasets is computationally intensive. One of the primary research directions for efficient training is to reduce trainin…
Speeding up NAS with Adaptive Subset Selection
Vishak Prasad C, Colin White, Paarth Jain +2
A majority of recent developments in neural architecture search (NAS) have been aimed at decreasing the computational cost of various techniques without affecting their final perfo…
Partitioned Gradient Matching-based Data Subset Selection for Compute-Efficient Robust ASR Training
Ashish Mittal, Durga Sivasubramanian, Rishabh Iyer +2
Training state-of-the-art ASR systems such as RNN-T often has a high associated financial and environmental cost. Training with a subset of training data could mitigate this proble…
AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning
Krishnateja Killamsetty, Guttu Sai Abhishek, Aakriti +4
Deep neural networks have seen great success in recent years; however, training a deep model is often challenging as its performance heavily depends on the hyper-parameters used. I…
BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets
Suraj Kothawade, Pavan Kumar Reddy, Ganesh Ramakrishnan +1
Current semi-supervised learning (SSL) methods assume a balance between the number of data points available for each class in both the labeled and the unlabeled data sets. However,…
Submodlib: A Submodular Optimization Library
Vishal Kaushal, Ganesh Ramakrishnan, Rishabh Iyer
Submodular functions are a special class of set functions which naturally model the notion of representativeness, diversity, coverage etc. and have been shown to be computationally…
PLATINUM: Semi-Supervised Model Agnostic Meta-Learning using Submodular Mutual Information
Changbin Li, Suraj Kothawade, Feng Chen +1
Few-shot classification (FSC) requires training models using a few (typically one to five) data points per class. Meta learning has proven to be able to learn a parametrized model…
GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning
Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer +1
Continual learning (CL) aims to develop techniques by which a single model adapts to an increasing number of tasks encountered sequentially, thereby potentially leveraging learning…
Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming
Ayush Maheshwari, Krishnateja Killamsetty, Ganesh Ramakrishnan +3
A critical bottleneck in supervised machine learning is the need for large amounts of labeled data which is expensive and time consuming to obtain. However, it has been shown that…
SPEAR : Semi-supervised Data Programming in Python
Guttu Sai Abhishek, Harshad Ingole, Parth Laturia +4
We present SPEAR, an open-source python library for data programming with semi supervision. The package implements several recent data programming approaches including facility to…
SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios
Suraj Kothawade, Nathan Beck, Krishnateja Killamsetty +1
Active learning has proven to be useful for minimizing labeling costs by selecting the most informative samples. However, existing active learning methods do not work well in reali…
Training Data Subset Selection for Regression with Controlled Generalization Error
Durga Sivasubramanian, Rishabh Iyer, Ganesh Ramakrishnan +1
Data subset selection from a large number of training instances has been a successful approach toward efficient and cost-effective machine learning. However, models trained on a sm…
RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning
Krishnateja Killamsetty, Xujiang Zhao, Feng Chen +1
Semi-supervised learning (SSL) algorithms have had great success in recent years in limited labeled data regimes. However, the current state-of-the-art SSL algorithms are computati…
Fair Machine Learning under Limited Demographically Labeled Data
Mustafa Safa Ozdayi, Murat Kantarcioglu, Rishabh Iyer
Research has shown that, machine learning models might inherit and propagate undesired social biases encoded in the data. To address this problem, fair training algorithms are deve…
Submodular Mutual Information for Targeted Data Subset Selection
Suraj Kothawade, Vishal Kaushal, Ganesh Ramakrishnan +2
With the rapid growth of data, it is becoming increasingly difficult to train or improve deep learning models with the right subset of data. We show that this problem can be effect…
GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training
Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan +2
The great success of modern machine learning models on large datasets is contingent on extensive computational resources with high financial and environmental costs. One way to add…
A Nested Bi-level Optimization Framework for Robust Few Shot Learning
Krishnateja Killamsetty, Changbin Li, Chen Zhao +2
Model-Agnostic Meta-Learning (MAML), a popular gradient-based meta-learning framework, assumes that the contribution of each task or instance to the meta-learner is equal. Hence, i…