A Tutorial on Principal Component Analysis

Abstract

Principal component analysis (PCA) is a mainstay of modern data analysis - a black box that is widely used but (sometimes) poorly understood. The goal of this paper is to dispel the magic behind this black box. This manuscript focuses on building a solid intuition for how and why principal component analysis works. This manuscript crystallizes this knowledge by deriving from simple intuitions, the mathematics behind PCA. This tutorial does not shy away from explaining the ideas informally, nor does it shy away from the mathematics. The hope is that by addressing both aspects, readers of all levels will be able to gain a better understanding of PCA as well as the when, the how and the why of applying this technique.

Cited by in corpus (96)

• On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice
• Machine Learning in Aerodynamic Shape Optimization
• Toward Unobtrusive In-home Gait Analysis Based on Radar Micro-Doppler Signatures
• Harnessing speckle for a sub-femtometre resolved broadband wavemeter and laser stabilization
• IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective
• Weighted principal component analysis: a weighted covariance eigendecomposition approach
• Analysis of the C${}_2$ (d${}^3{\Pi }_g$-a${}^3{\Pi }_u$) Swan bands as a thermometric probe in CO${}_2$ microwave plasmas
• PCA meets RG
• Exploring the Space of Black-box Attacks on Deep Neural Networks
• Unsupervised machine learning for detection of phase transitions in off-lattice systems I. Foundations
• Characterizing the Decision Boundary of Deep Neural Networks
• Unsupervised machine learning for detection of phase transitions in off-lattice systems II. Applications
• Adversarial Examples - A Complete Characterisation of the Phenomenon
• A random version of principal component analysis in data clustering
• Global-scale massive feature extraction from monthly hydroclimatic time series: Statistical characterizations, spatial patterns and hydrological similarity
• Stellar classification from single-band imaging using machine learning
• Reducing the complexity of chemical networks via interpretable autoencoders
• Robust PCA for Anomaly Detection in Cyber Networks
• The cosmic ray muon tomography facility based on large scale MRPC detector
• SDCOR: Scalable Density-based Clustering for Local Outlier Detection in Massive-Scale Datasets
• Structured and Unstructured Outlier Identification for Robust PCA: A Non iterative, Parameter free Algorithm
• Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering
• Galaxy classification: deep learning on the OTELO and COSMOS databases
• A Review of Nonnegative Matrix Factorization Methods for Clustering
• A Bayesian method for detecting stellar flares
• Simulated evolution of protein-protein interaction networks with realistic topology
• Implementation and comparative quantitative assessment of different multispectral image pansharpening approches
• Exploratory Data Analysis for Airline Disruption Management
• Astronomaly at scale: searching for anomalies amongst 4 million galaxies
• Autoencoder, Principal Component Analysis and Support Vector Regression for Data Imputation
• Unsupervised classification of simulated magnetospheric regions
• Field theoretical approach for signal detection in nearly continuous positive spectra II: Tensorial data
• Parameter Estimation with Maximal Updated Densities
• Simple and Effective Dimensionality Reduction for Word Embeddings
• Dev2vec: Representing Domain Expertise of Developers in an Embedding Space
• Using PCA to Efficiently Represent State Spaces
• Second largest Eigenpair Statistics for Sparse Graphs
• Multiscale principal component analysis
• Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
• Artificial Intelligence: A Child's Play
• Extraction of clinical information from the non-invasive fetal electrocardiogram
• Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit
• Principal component analysis within nuclear structure
• Microstructure under the Microscope: Tools to Survive and Thrive in The Age of (Too Much) Information
• High-precision programming of large-scale ring resonator circuits with minimal pre-calibration
• Digital representation and quantification of discrete dislocation networks
• Comparison of Data Imputation Techniques and their Impact
• Low-rank and Sparse Soft Targets to Learn Better DNN Acoustic Models
• Clustering in Recurrent Neural Networks for Micro-Segmentation using Spending Personality
• Towards Exploratory Landscape Analysis for Large-scale Optimization: A Dimensionality Reduction Framework
• Comparing directed networks via denoising graphlet distributions
• Phishing Detection through Email Embeddings
• Ambitions for theory in the physics of life
• BSpell: A CNN-Blended BERT Based Bangla Spell Checker
• Learning phase transitions: comparing PCA and SVM
• Smart Analytical Signature Verification For DSP Applications
• PCA Method for Automated Detection of Mispronounced Words
• Dynamic Principal Component Analysis: Identifying the Relationship between Multiple Air Pollutants
• Attacker Behaviour Profiling using Stochastic Ensemble of Hidden Markov Models
• Redshift determination through weighted phase correlation: a linearithmic implementation
• Implicit LOD using points ordering for processing and visualisation in Point Cloud Servers
• DigiVoice: Voice Biomarker Featurization and Analysis Pipeline
• A Data Analytics Perspective of the Clarke and Related Transforms in Power Grid Analysis
• Datasets of the solar quiet (Sq) and solar disturbed (SD) variations of the geomagnetic field from the mid latitudinal Magnetic Observatory of Coimbra (Portugal) obtained by different methods
• A Survey of Techniques All Classifiers Can Learn from Deep Networks: Models, Optimizations, and Regularization
• TESDA: Transform Enabled Statistical Detection of Attacks in Deep Neural Networks
• Opening the low-background and high-spectral-resolution domain with the ATHENA large X-ray observatory: Development of the Cryogenic AntiCoincidence Detector for the X-ray Integral Field Unit
• Subjective Question Answering: Deciphering the inner workings of Transformers in the realm of subjectivity
• Principal Component Analysis: Resources for an Essential Application of Linear Algebra
• Beam Profiling with Noise Reduction From Computer Vision and Principal Component Analysis for the MAGIS-100 Experiment
• Fast, Parameter free Outlier Identification for Robust PCA
• A New Approach for Scalable Analysis of Microbial Communities
• ToF-SIMS Investigations on Dental Implant Materials and Adsorbed Protein Films
• Out-Of-Distribution Detection With Subspace Techniques And Probabilistic Modeling Of Features
• Optimization of Wireless Sensor Network Deployment for Spatiotemporal Reconstruction and Prediction
• Localized Compression: Applying Convolutional Neural Networks to Compressed Images
• Quantum algorithm for the classification of Supersymmetric top quark events
• Vis-SPLIT: Interactive Hierarchical Modeling for mRNA Expression Classification
• Interpretable Network Representation Learning with Principal Component Analysis
• Comparison of Clustering Methods for Extraction of Uncorrelated Sparse Sources from Data Mixtures
• Algorithms and Hardness for Linear Algebra on Geometric Graphs
• TransAug: Translate as Augmentation for Sentence Embeddings
• Unsupervised Spoken Term Discovery on Untranscribed Speech
• Learning Low-dimensional Manifolds for Scoring of Tissue Microarray Images
• Towards real time assessment of intramuscular fat content in meat using optical fibre-based optical coherence tomography
• Input-independent Attention Weights Are Expressive Enough: A Study of Attention in Self-supervised Audio Transformers
• RPM-Net: Robust Pixel-Level Matching Networks for Self-Supervised Video Object Segmentation
• Progressive Compressed Records: Taking a Byte out of Deep Learning Data
• Analysis and Understanding of Various Models for Efficient Representation and Accurate Recognition of Human Faces
• Multi-Fusion Chinese WordNet (MCW) : Compound of Machine Learning and Manual Correction
• Enhancing keyword correlation for event detection in social networks using SVD and k-means: Twitter case study
• An octree cells occupancy geometric dimensionality descriptor for massive on-server point cloud visualisation and classification
• A Repeated Signal Difference for Recognising Patterns
• Comparison of feature extraction and dimensionality reduction methods for single channel extracellular spike sorting
• Homogenization of Metamaterials by Dual Interpolation of Fields: a Rigorous Treatment of Resonances and Nonlocality
• Liquid Sensing Using WiFi Signals