4 papers
Unlearning with Asymmetric Sources: Improved Unlearning-Utility Trade-off with Public Data
Ahmed Mehdi Inane, Vincent Quirion, Gintare Karolina Dziugaite +1
Noise-based certified machine unlearning currently faces a hard ceiling: the noise magnitude required to certify unlearning typically destroys model utility, particularly for large…
A Derandomization Framework for Structure Discovery: Applications in Neural Networks and Beyond
Nikos Tsikouras, Yorgos Pantis, Ioannis Mitliagkas +1
Understanding the dynamics of feature learning in neural networks (NNs) remains a significant challenge. The work of (Mousavi-Hosseini et al., 2023) analyzes a multiple index teach…
Orth-Dion: Eliminating Geometric Mismatch in Distributed Low-Rank Spectral Optimization
Tatsuhiro Nakamori, Laura Gomezjurado Gonzalez, Ganesh Talluri +5
Low-rank gradient compression reduces communication in distributed training by representing updates with rank- factors. Dion is a recent method that approximates Muon, a spectra…
Feature learning as alignment: a structural property of gradient descent in non-linear neural networks
Daniel Beaglehole, Ioannis Mitliagkas, Atish Agarwala
Understanding the mechanisms through which neural networks extract statistics from input-label pairs through feature learning is one of the most important unsolved problems in supe…