5 papers · 1 filter
Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates
Parjanya Prajakta Prashant, Jiongli Zhu, Aldan Creo +1
Fine-tuning large language models on new data improves task performance but degrades capabilities learned during pretraining, a phenomenon known as catastrophic forgetting. Existin…
Scalable Out-of-distribution Robustness in the Presence of Unobserved Confounders
Parjanya Prashant, Seyedeh Baharan Khatami, Bruno Ribeiro +1
We consider the task of out-of-distribution (OOD) generalization, where the distribution shift is due to an unobserved confounder () affecting both the covariates () and the…
KAIROS: Scalable Model-Agnostic Data Valuation
Jiongli Zhu, Parjanya Prajakta Prashant, Alex Cloninger +1
Training data increasingly shapes not only model accuracy but also regulatory compliance and market valuation of AI assets. Yet existing valuation methods remain inadequate: model-…
A Lightweight Method to Disrupt Memorized Sequences in LLM
Parjanya Prajakta Prashant, Kaustubh Ponkshe, Babak Salimi
As language models scale, their performance improves dramatically across a wide range of tasks, but so does their tendency to memorize and regurgitate parts of their training data…
Differentiable Causal Discovery For Latent Hierarchical Causal Models
Parjanya Prashant, Ignavier Ng, Kun Zhang +1
Discovering causal structures with latent variables from observational data is a fundamental challenge in causal discovery. Existing methods often rely on constraint-based, iterati…