Randomized Nonlinear Component Analysis
arXiv:1402.0119
Abstract
Classical methods such as Principal Component Analysis (PCA) and Canonical Correlation Analysis (CCA) are ubiquitous in statistics. However, these techniques are only able to reveal linear relationships in data. Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale. In a separate strand of recent research, randomized methods have been proposed to construct features that help reveal nonlinear patterns in data. For basic tasks such as regression or classification, random features exhibit little or no loss in performance, while achieving drastic savings in computational requirements. In this paper we leverage randomness to design scalable new variants of nonlinear PCA and CCA; our ideas extend to key multivariate analysis tools such as spectral clustering or LDA. We demonstrate our algorithms through experiments on real-world data, on which we compare against the state-of-the-art. A simple R implementation of the presented algorithms is provided.
Appearing in ICML 2014
References in corpus (2)
Cited by in corpus (24)
- A Survey of Multi-View Representation Learning
- Relational Autoencoder for Feature Extraction
- Large-Scale Kernel Methods for Independence Testing
- Graph Multiview Canonical Correlation Analysis
- Nonparametric Canonical Correlation Analysis
- Canonical Correlation Analysis (CCA) Based Multi-View Learning: An Overview
- Randomized ICA and LDA Dimensionality Reduction Methods for Hyperspectral Image Classification
- SPSD Matrix Approximation vis Column Selection: Theories, Algorithms, and Extensions
- Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond
- A Randomized Algorithm for CCA
- Nonlinear Multiview Analysis: Identifiability and Neural Network-assisted Implementation
- Scale Up Nonlinear Component Analysis with Doubly Stochastic Gradients
- Approximate Kernel PCA Using Random Features: Computational vs. Statistical Trade-off
- Fast Learning in Reproducing Kernel Krein Spaces via Signed Measures
- Simple and Almost Assumption-Free Out-of-Sample Bound for Random Feature Mapping
- Variational Inference for Deep Probabilistic Canonical Correlation Analysis
- Towards a Unified Quadrature Framework for Large-Scale Kernel Machines
- Robust Matrix Elastic Net based Canonical Correlation Analysis: An Effective Algorithm for Multi-View Unsupervised Learning
- Statistical Optimality and Computational Efficiency of Nyström Kernel PCA
- Large-scale Kernel-based Feature Extraction via Budgeted Nonlinear Subspace Tracking
- ORCCA: Optimal Randomized Canonical Correlation Analysis
- On Dimension-free Tail Inequalities for Sums of Random Matrices and Applications
- Tensor Canonical Correlation Analysis with Convergence and Statistical Guarantees
- Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics