4 papers · 1 filter
Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates
Parjanya Prajakta Prashant, Jiongli Zhu, Aldan Creo +1
Fine-tuning large language models on new data improves task performance but degrades capabilities learned during pretraining, a phenomenon known as catastrophic forgetting. Existin…
KAIROS: Scalable Model-Agnostic Data Valuation
Jiongli Zhu, Parjanya Prajakta Prashant, Alex Cloninger +1
Training data increasingly shapes not only model accuracy but also regulatory compliance and market valuation of AI assets. Yet existing valuation methods remain inadequate: model-…
A Lightweight Method to Disrupt Memorized Sequences in LLM
Parjanya Prajakta Prashant, Kaustubh Ponkshe, Babak Salimi
As language models scale, their performance improves dramatically across a wide range of tasks, but so does their tendency to memorize and regurgitate parts of their training data…
OTClean: Data Cleaning for Conditional Independence Violations using Optimal Transport
Alireza Pirhadi, Mohammad Hossein Moslemi, Alexander Cloninger +2
Ensuring Conditional Independence (CI) constraints is pivotal for the development of fair and trustworthy machine learning models. In this paper, we introduce \sys, a framework tha…