Stochastic Approximation for Canonical Correlation Analysis

Abstract

We propose novel first-order stochastic approximation algorithms for canonical correlation analysis (CCA). Algorithms presented are instances of inexact matrix stochastic gradient (MSG) and inexact matrix exponentiated gradient (MEG), and achieve $\epsilon$-suboptimality in the population objective in $\mathrm{poly}(\frac{1}{\epsilon })$ iterations. We also consider practical variants of the proposed algorithms and compare them with other methods for CCA both theoretically and empirically.

Cited by in corpus (8)

• Stochastic Canonical Correlation Analysis
• On Landscape of Lagrangian Functions and Stochastic Search for Constrained Nonconvex Optimization
• A simple normative network approximates local non-Hebbian learning in the cortex
• Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problems
• Representation Learning for Words and Entities
• ORCCA: Optimal Randomized Canonical Correlation Analysis
• Gen-Oja: A Two-time-scale approach for Streaming CCA
• Tensor Canonical Correlation Analysis with Convergence and Statistical Guarantees