4 citations · 6 across the 17 of their papers we have counts for
15 papers · 1 filter
Unlearning Deceptive Behaviors in LLMs with Contrastive Forget Sets
Haoran Tang, Rajiv Khanna
Large language models often know the truth and say otherwise: a model that answers correctly when asked neutrally will affirm a user's mistaken belief, or misstate a fact its syste…
Unmerge: Efficient Machine Unlearning via Task Arithmetic
Haoran Tang, Andrew Tan, Rajiv Khanna
Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hy…
When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay
Hasan Amin, Wei-Kai Chang, Rajiv Khanna
Normalization renders large parts of neural networks effectively scale invariant, inducing a hidden feedback loop in which learning-rate schedules and weight decay interact through…
Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization
Hasan Amin, Kian Ahrabian, Ming Yin +1
Modern language-model fine-tuning typically pairs each prompt with a single response, even though many prompts admit multiple valid completions. This effectively reduces a multi-mo…
Consistent Diffusion Language Models
Hasan Amin, Yuan Gao, Yaser Souri +4
Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive…
Why Some Models Resist Unlearning: A Linear Stability Perspective
Wei-Kai Chang, Rajiv Khanna
Machine unlearning, the ability to erase the effect of specific training samples without retraining from scratch, is critical for privacy, regulation, and efficiency. However, most…
Contrastive Representation Shaping for LLM Unlearning
Haoran Tang, Rajiv Khanna
Most LLM unlearning methods aim to approximate retrain-from-scratch behaviors with minimal distribution shift, often via alignment-style objectives defined in the prediction space.…
Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes
Wei-Kai Chang, Rajiv Khanna
As deep learning models continue to scale, the growing computational demands have amplified the need for effective coreset selection techniques. Coreset selection aims to accelerat…
A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias
Wei-Kai Chang, Rajiv Khanna
Understanding the dynamics of optimization in deep learning is increasingly important as models scale. While stochastic gradient descent (SGD) and its variants reliably find soluti…
SAMOSA: Sharpness Aware Minimization for Open Set Active learning
Young In Kim, Andrea Agiollo, Rajiv Khanna
Modern machine learning solutions require extensive data collection where labeling remains costly. To reduce this burden, open set active learning approaches aim to select informat…
Structure-Aware Spectral Sparsification via Uniform Edge Sampling
Kaiwen He, Petros Drineas, Rajiv Khanna
Spectral clustering is a fundamental method for graph partitioning, but its reliance on eigenvector computation limits scalability to massive graphs. Classical sparsification metho…
Sharpness-Aware Machine Unlearning
Haoran Tang, Rajiv Khanna
We characterize the effectiveness of Sharpness-aware minimization (SAM) under machine unlearning scheme, where unlearning forget signals interferes with learning retain signals. Wh…
A Precise Characterization of SGD Stability Using Loss Surface Geometry
Gregory Dexter, Borja Ocejo, Sathiya Keerthi +3
Stochastic Gradient Descent (SGD) stands as a cornerstone optimization algorithm with proven real-world empirical successes but relatively limited theoretical understanding. Recent…
Membership Privacy Risks of Sharpness Aware Minimization
Young In Kim, Andrea Agiollo, Pratiksha Agrawal +2
Optimization algorithms that seek flatter minima, such as Sharpness-Aware Minimization (SAM), are credited with improved generalization and robustness to noise. We ask whether such…
Feature Space Sketching for Logistic Regression
Gregory Dexter, Rajiv Khanna, Jawad Raheel +1
We present novel bounds for coreset construction, feature selection, and dimensionality reduction for logistic regression. All three approaches can be thought of as sketching the l…