Linear and Quadratic Discriminant Analysis: Tutorial

Abstract

This tutorial explains Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) as two fundamental classification methods in statistical and probabilistic learning. We start with the optimization of decision boundary on which the posteriors are equal. Then, LDA and QDA are derived for binary and multiple classes. The estimation of parameters in LDA and QDA are also covered. Then, we explain how LDA and QDA are related to metric learning, kernel principal component analysis, Mahalanobis distance, logistic regression, Bayes optimal classifier, Gaussian naive Bayes, and likelihood ratio test. We also prove that LDA and Fisher discriminant analysis are equivalent. We finally clarify some of the theoretical concepts with simulations we provide.

16 pages, 7 figures. v2: corrected some mathematical typos

References in corpus (3)

• Eigenvalue and Generalized Eigenvalue Problems: Tutorial
• Recognizing Involuntary Actions from 3D Skeleton Data Using Body States
• Fitting A Mixture Distribution to Data: Tutorial

Cited by in corpus (5)

• Locally Linear Embedding and its Variants: Tutorial and Survey
• Fisher and Kernel Fisher Discriminant Analysis: Tutorial
• A Novel Use of Discrete Wavelet Transform Features in the Prediction of Epileptic Seizures from EEG Data
• Roweisposes, Including Eigenposes, Supervised Eigenposes, and Fisherposes, for 3D Action Recognition
• Weighted Fisher Discriminant Analysis in the Input and Feature Spaces