Eigenvalue and Generalized Eigenvalue Problems: Tutorial

Abstract

This paper is a tutorial for eigenvalue and generalized eigenvalue problems. We first introduce eigenvalue problem, eigen-decomposition (spectral decomposition), and generalized eigenvalue problem. Then, we mention the optimization problems which yield to the eigenvalue and generalized eigenvalue problems. We also provide examples from machine learning, including principal component analysis, kernel supervised principal component analysis, and Fisher discriminant analysis, which result in eigenvalue and generalized eigenvalue problems. Finally, we introduce the solutions to both eigenvalue and generalized eigenvalue problems.

8 pages, Tutorial paper. v2, v3: Added additional information

Cited by in corpus (23)

• Linear and Quadratic Discriminant Analysis: Tutorial
• Feature Selection and Feature Extraction in Pattern Analysis: A Literature Review
• Fisher Discriminant Triplet and Contrastive Losses for Training Siamese Networks
• Locally Linear Embedding and its Variants: Tutorial and Survey
• Multidimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey
• Unsupervised and Supervised Principal Component Analysis: Tutorial
• Fisher and Kernel Fisher Discriminant Analysis: Tutorial
• Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nyström Method, and Use of Kernels in Machine Learning: Tutorial and Survey
• Roweis Discriminant Analysis: A Generalized Subspace Learning Method
• EigenGame: PCA as a Nash Equilibrium
• Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey
• Dynamic Metasurface Antennas for MIMO-OFDM Receivers with Bit-Limited ADCs
• Similarity-and-Independence-Aware Beamformer with Iterative Casting and Boost Start for Target Source Extraction Using Reference
• Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey
• Capturing patterns of variation unique to a specific dataset
• A Bayesian Fisher-EM algorithm for discriminative Gaussian subspace clustering
• Quantized Fisher Discriminant Analysis
• Unified Framework for Spectral Dimensionality Reduction, Maximum Variance Unfolding, and Kernel Learning By Semidefinite Programming: Tutorial and Survey
• Generative Locally Linear Embedding
• Sufficient Dimension Reduction for High-Dimensional Regression and Low-Dimensional Embedding: Tutorial and Survey
• Weighted Fisher Discriminant Analysis in the Input and Feature Spaces
• Dynamic Oversampling Tecniques for 1-Bit ADCs in Large-Scale MIMO Systems
• Roweisposes, Including Eigenposes, Supervised Eigenposes, and Fisherposes, for 3D Action Recognition