GSVD for Geometry-Grounded Dataset Comparison: An Alignment Angle Is All You Need

Abstract

Geometry-grounded learning asks models to respect structure in the problem domain rather than treating observations as arbitrary vectors. Motivated by this view, we revisit a classical but underused primitive for comparing datasets: linear relations between two data matrices, expressed via the co-span constraint $Ax=By=z$ in a shared ambient space. To operationalize this comparison, we use the generalized singular value decomposition (GSVD) as a joint coordinate system for two subspaces. In particular, we exploit the GSVD form $A=HCU$, $B=HSV$ with $C^{\top }C+S^{\top }S=I$, which separates shared versus dataset-specific directions through the diagonal structure of $(C,S)$. From these factors we derive an interpretable *angle score* $\hat{I}¸(z)\in [0,\ddot{I}€/2]$ for a sample $z$, quantifying whether z is explained relatively more by $A$, more by $B$, or comparably by both. The primary role of $\hat{I}¸(z)$ is as a *per-sample geometric diagnostic*. We illustrate the behavior of the score on MNIST through angle distributions and representative GSVD directions. A binary classifier derived from $\hat{I}¸(z)$ is presented as an illustrative application of the score as an interpretable diagnostic tool.

20 pages, GRaM workshop ICLR 2026