38 citations · 38 across the 5 of their papers we have counts for
5 papers · 1 filter
Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness
Siqiao Mu, Diego Klabjan
We establish the first convergence guarantees of gradient descent for general feedforward neural networks of any width or depth, with any initialization or dataset. We only assume…
On the Convergence Rate of LoRA Gradient Descent
Siqiao Mu, Diego Klabjan
The low-rank adaptation (LoRA) algorithm for fine-tuning large models has grown popular in recent years due to its remarkable performance and low computational requirements. LoRA t…
Descend or Rewind? Stochastic Gradient Descent Unlearning
Siqiao Mu, Diego Klabjan
Machine unlearning algorithms aim to remove the impact of selected training data from a model without the computational expenses of retraining from scratch. Two such algorithms are…
Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions
Siqiao Mu, Diego Klabjan
Machine unlearning algorithms aim to efficiently remove data from a model without retraining it from scratch, in order to remove corrupted or outdated data or respect a user's ``ri…
On the Second-Order Convergence of Biased Policy Gradient Algorithms
Siqiao Mu, Diego Klabjan
Since the objective functions of reinforcement learning problems are typically highly nonconvex, it is desirable that policy gradient, the most popular algorithm, escapes saddle po…