10 papers
Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness
Siqiao Mu, Diego Klabjan
We establish convergence guarantees of gradient descent for general feedforward neural networks of arbitrary width or depth, with no special requirements on the initialization or d…
Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learning
Haemin Park, Diego Klabjan, Martin W. Braun +2
Class imbalance poses a critical challenge in federated learning (FL), where underrepresented classes suffer from poor predictive performance yet cannot be addressed by standard ce…
Hierarchical Graph Learning for Calendar Spread Strategies in Commodity Futures Markets
Yoonsik Hong, Diego Klabjan
Commodity futures can be represented hierarchically, with underlying assets at the upper level and individual futures contracts at the lower level. Entities at each level can be co…
On the Convergence Rate of LoRA Gradient Descent
Siqiao Mu, Diego Klabjan
The low-rank adaptation (LoRA) algorithm for fine-tuning large models has grown popular in recent years due to its remarkable performance and low computational requirements. LoRA t…
Descend or Rewind? Stochastic Gradient Descent Unlearning
Siqiao Mu, Diego Klabjan
Machine unlearning algorithms aim to remove the impact of selected training data from a model without the computational expenses of retraining from scratch. Two such algorithms are…
Rank-Accuracy Trade-off for LoRA: A Gradient-Flow Analysis
Michael Rushka, Diego Klabjan
Previous empirical studies have shown that LoRA achieves accuracy comparable to full-parameter methods on downstream fine-tuning tasks, even for rank-1 updates. By contrast, the th…