From Gradient Descent to Potential Theory: A Geometric Dictionary for Binary Classification

Abstract

We propose a dictionary between binary classification in machine learning and classical potential theory on vector bundles. Classifiers are parallel sections of vector bundles over the data space; training labels become Dirichlet boundary conditions; the kernel of an RKHS interpolant is the Green's function of an elliptic operator; and backpropagation is the flat-geometry limit of an exact geometric problem. The central identification: for L = (Delta + kappa^2)^nu with nu > n/2, the L-harmonic interpolation problem - find the minimum-H^nu-norm classifier satisfying Lf = 0 away from the data with prescribed values at the training points - is precisely what RKHS interpolation and kernel ridge regression already solve. Kimeldorf-Wahba (1971) and Lindgren-Rue-Lindstrom (2011) supply the analytic content; what we add is the potential-theoretic reading: the kernel is the Green's function of L, the coefficients are electrostatic capacitances, and the decision boundary is the zero equipotential. The condition nu > n/2 excludes the Dirichlet energy (nu = 1) in dimension n >= 2, where point data has zero capacity. The hard-margin SVM is a margin-constrained variant. For finite data on any smooth manifold, flat O(2) solutions always exist. Over non-contractible bases the geometry becomes load-bearing: the parity invariant on S^1 and the Euler class on S^2 are obstructions no classifier can evade. Code is available on GitHub.