6 citations · 6 across the 9 of their papers we have counts for
11 papers
SoftServe: A Scalable Quasi-Newton Method for Deep Learning
Joohwan Ko, Tetiana Parshakova, Diana Cai +1
Quasi-Newton (QN) methods have long been among the most effective methods for large-scale unconstrained convex optimization. Two obstacles have limited their use in deep learning:…
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer
Nikhil Ghosh, Tetiana Parshakova, Robert M. Gower
Low-rank adaptation (LoRA) makes finetuning large language models cheaper by adding to each weight matrix a trainable low-rank update parameterized as the product of two matrices.…
Muon Does Not Converge on Convex Lipschitz Functions
Tetiana Parshakova, Ahmed Khaled, Michael Crawshaw +2
Muon and its variants have shown strong empirical performance in a variety of deep learning tasks. Existing convergence analyses of Muon rely on smoothness assumptions, though argu…
Multiple Approximate-Response Agents (MARA): Fast Near-Optimal Primal Recovery for Distributed Optimization
Tetiana Parshakova, Yicheng Bai, Garrett van Ryzin +1
Dual methods are useful for distributed optimization because they allow agent-level subproblems to be solved in parallel. However, achieving primal feasibility with dual methods is…
Optimization Algorithm Design via Electric Circuits
Stephen P. Boyd, Tetiana Parshakova, Ernest K. Ryu +1
We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits. Given an optimization problem, the first stage of the methodology is…
Fitting Multilevel Factor Models
Tetiana Parshakova, Trevor Hastie, Stephen Boyd
We examine a special case of the multilevel factor model, with covariance given by multilevel low rank (MLR) matrix~\cite{parshakova2023factor}. We develop a novel, fast implementa…
Factor Fitting, Rank Allocation, and Partitioning in Multilevel Low Rank Matrices
Tetiana Parshakova, Trevor Hastie, Eric Darve +1
We consider multilevel low rank (MLR) matrices, defined as a row and column permutation of a sum of matrices, each one a block diagonal refinement of the previous one, with all blo…
Efficient Graph Field Integrators Meet Point Clouds
Krzysztof Choromanski, Arijit Sehanobish, Han Lin +13
We present two new classes of algorithms for efficient field integration on graphs encoding point clouds. The first class, SeparatorFactorization(SF), leverages the bounded genus o…
Implementation of an Oracle-Structured Bundle Method for Distributed Optimization
Tetiana Parshakova, Fangzhao Zhang, Stephen Boyd
We consider the problem of minimizing a function that is a sum of convex agent functions plus a convex common public function that couples them. The agent functions can only be acc…
Distributional Reinforcement Learning for Energy-Based Sequential Models
Tetiana Parshakova, Jean-Marc Andreoli, Marc Dymetman
Global Autoregressive Models (GAMs) are a recent proposal [Parshakova et al., CoNLL 2019] for exploiting global properties of sequences for data-efficient learning of seq2seq model…
Global Autoregressive Models for Data-Efficient Sequence Learning
Tetiana Parshakova, Jean-Marc Andreoli, Marc Dymetman
Standard autoregressive seq2seq models are easily trained by max-likelihood, but tend to show poor results under small-data conditions. We introduce a class of seq2seq models, GAMs…