2 citations · 2 across the 8 of their papers we have counts for
4 papers · 1 filter
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate
Zhiqi Bu, Xiaomeng Jin, Bhanukiran Vinzamuri +4
Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspectiv…
EMC: Efficient MCMC Negative Sampling for Contrastive Learning with Global Convergence
Chung-Yiu Yau, Hoi-To Wai, Parameswaran Raman +2
A key challenge in contrastive learning is to generate negative samples from a large sample set to contrast with positive samples, for learning better encoding of the data. These n…
Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
Ruichen Jiang, Parameswaran Raman, Shoham Sabach +3
Second-order optimization methods, such as cubic regularized Newton methods, are known for their rapid convergence rates; nevertheless, they become impractical in high-dimensional…
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
Kaan Ozkara, Can Karakus, Parameswaran Raman +4
Following the introduction of Adam, several novel adaptive optimizers for deep learning have been proposed. These optimizers typically excel in some tasks but may not outperform Ad…