6 citations · 6 across the 4 of their papers we have counts for
8 papers
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer
Nikhil Ghosh, Tetiana Parshakova, Robert M. Gower
Low-rank adaptation (LoRA) makes finetuning large language models cheaper by adding to each weight matrix a trainable low-rank update parameterized as the product of two matrices.…
Muon Does Not Converge on Convex Lipschitz Functions
Tetiana Parshakova, Ahmed Khaled, Michael Crawshaw +2
Muon and its variants have shown strong empirical performance in a variety of deep learning tasks. Existing convergence analyses of Muon rely on smoothness assumptions, though argu…
Multiple Approximate-Response Agents (MARA): Fast Near-Optimal Primal Recovery for Distributed Optimization
Tetiana Parshakova, Yicheng Bai, Garrett van Ryzin +1
Dual methods are useful for distributed optimization because they allow agent-level subproblems to be solved in parallel. However, achieving primal feasibility with dual methods is…
Optimization Algorithm Design via Electric Circuits
Stephen P. Boyd, Tetiana Parshakova, Ernest K. Ryu +1
We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits. Given an optimization problem, the first stage of the methodology is…
Fitting Multilevel Factor Models
Tetiana Parshakova, Trevor Hastie, Stephen Boyd
We examine a special case of the multilevel factor model, with covariance given by multilevel low rank (MLR) matrix~\cite{parshakova2023factor}. We develop a novel, fast implementa…
Factor Fitting, Rank Allocation, and Partitioning in Multilevel Low Rank Matrices
Tetiana Parshakova, Trevor Hastie, Eric Darve +1
We consider multilevel low rank (MLR) matrices, defined as a row and column permutation of a sum of matrices, each one a block diagonal refinement of the previous one, with all blo…