4 citations · 6 across the 17 of their papers we have counts for
24 papers
Learning What to Forget: Distributional Unlearning for LLM Representation Spaces
Pinaki Mohanty, Haoran Tang, Maggie Makar +1
Machine learning systems increasingly face the need to remove the influence of entire data domains, such as toxic language, harmful behavior, or topical content, rather than isolat…
Unlearning Deceptive Behaviors in LLMs with Contrastive Forget Sets
Haoran Tang, Rajiv Khanna
Large language models often know the truth and say otherwise: a model that answers correctly when asked neutrally will affirm a user's mistaken belief, or misstate a fact its syste…
Unmerge: Efficient Machine Unlearning via Task Arithmetic
Haoran Tang, Andrew Tan, Rajiv Khanna
Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hy…
Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
Hasan Amin, Ming Yin, Rajiv Khanna
Diffusion language models (DLMs) promise fast parallel generation, yet high-quality samples often require large number of refinement steps, which diminishes their advantage in prac…
When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay
Hasan Amin, Wei-Kai Chang, Rajiv Khanna
Normalization renders large parts of neural networks effectively scale invariant, inducing a hidden feedback loop in which learning-rate schedules and weight decay interact through…
Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization
Hasan Amin, Kian Ahrabian, Ming Yin +1
Modern language-model fine-tuning typically pairs each prompt with a single response, even though many prompts admit multiple valid completions. This effectively reduces a multi-mo…
Consistent Diffusion Language Models
Hasan Amin, Yuan Gao, Yaser Souri +4
Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive…
From Fallback to Frontline: When Can LLMs be Superior Annotators of Human Perspectives?
Hasan Amin, Harry Yizhou Tian, Xiaoni Duan +3
Although large language models (LLMs) are increasingly used as annotators at scale, they are typically treated as a pragmatic fallback rather than a faithful estimator of human per…
Align When They Want, Complement When They Need! Human-Centered Ensembles for Adaptive Human-AI Collaboration
Hasan Amin, Ming Yin, Rajiv Khanna
In human-AI decision making, designing AI that complements human expertise has been a natural strategy to enhance human-AI collaboration, yet it often comes at the cost of decrease…
Why Some Models Resist Unlearning: A Linear Stability Perspective
Wei-Kai Chang, Rajiv Khanna
Machine unlearning, the ability to erase the effect of specific training samples without retraining from scratch, is critical for privacy, regulation, and efficiency. However, most…
Contrastive Representation Shaping for LLM Unlearning
Haoran Tang, Rajiv Khanna
Most LLM unlearning methods aim to approximate retrain-from-scratch behaviors with minimal distribution shift, often via alignment-style objectives defined in the prediction space.…
Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes
Wei-Kai Chang, Rajiv Khanna
As deep learning models continue to scale, the growing computational demands have amplified the need for effective coreset selection techniques. Coreset selection aims to accelerat…
A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias
Wei-Kai Chang, Rajiv Khanna
Understanding the dynamics of optimization in deep learning is increasingly important as models scale. While stochastic gradient descent (SGD) and its variants reliably find soluti…
SAMOSA: Sharpness Aware Minimization for Open Set Active learning
Young In Kim, Andrea Agiollo, Rajiv Khanna
Modern machine learning solutions require extensive data collection where labeling remains costly. To reduce this burden, open set active learning approaches aim to select informat…
Structure-Aware Spectral Sparsification via Uniform Edge Sampling
Kaiwen He, Petros Drineas, Rajiv Khanna
Spectral clustering is a fundamental method for graph partitioning, but its reliance on eigenvector computation limits scalability to massive graphs. Classical sparsification metho…
Sharpness-Aware Machine Unlearning
Haoran Tang, Rajiv Khanna
We characterize the effectiveness of Sharpness-aware minimization (SAM) under machine unlearning scheme, where unlearning forget signals interferes with learning retain signals. Wh…
The Space Complexity of Approximating Logistic Loss
Gregory Dexter, Petros Drineas, Rajiv Khanna
We provide space complexity lower bounds for data structures that approximate logistic loss up to -relative error on a logistic regression problem with data $\mathbf{X} \in \mat…
A Precise Characterization of SGD Stability Using Loss Surface Geometry
Gregory Dexter, Borja Ocejo, Sathiya Keerthi +3
Stochastic Gradient Descent (SGD) stands as a cornerstone optimization algorithm with proven real-world empirical successes but relatively limited theoretical understanding. Recent…
Membership Privacy Risks of Sharpness Aware Minimization
Young In Kim, Andrea Agiollo, Pratiksha Agrawal +2
Optimization algorithms that seek flatter minima, such as Sharpness-Aware Minimization (SAM), are credited with improved generalization and robustness to noise. We ask whether such…
Generalization Guarantees via Algorithm-dependent Rademacher Complexity
Sarah Sachs, Tim van Erven, Liam Hodgkinson +2
Algorithm- and data-dependent generalization bounds are required to explain the generalization behavior of modern machine learning algorithms. In this context, there exists informa…
Feature Space Sketching for Logistic Regression
Gregory Dexter, Rajiv Khanna, Jawad Raheel +1
We present novel bounds for coreset construction, feature selection, and dimensionality reduction for logistic regression. All three approaches can be thought of as sketching the l…
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization
Kayhan Behdin, Qingquan Song, Aman Gupta +7
Modern deep learning models are over-parameterized, where different optima can result in widely varying generalization performance. The Sharpness-Aware Minimization (SAM) technique…
Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers
Liam Hodgkinson, Umut Şimşekli, Rajiv Khanna +1
Despite the ubiquitous use of stochastic optimization algorithms in machine learning, the precise impact of these algorithms and their dynamics on generalization performance in rea…
LocalNewton: Reducing Communication Bottleneck for Distributed Learning
Vipul Gupta, Avishek Ghosh, Michal Derezinski +3
To address the communication bottleneck problem in distributed optimization within a master-worker framework, we propose LocalNewton, a distributed second-order algorithm with loca…