Rows vs. Columns: Randomized Kaczmarz or Gauss-Seidel for Ridge Regression

Ahmed Hefny, Deanna Needell, Aaditya Ramdas

The Kaczmarz and Gauss-Seidel methods aim to solve a linear $m\times n$ system $\mathbf{X}\mathbf{\beta }=\mathbf{y}$ by iteratively refining the solution estimate; the fo…