Generalized Tangent Kernel: A Unified Geometric Foundation for Natural Gradient and Standard Gradient

Qinxun Bai, Steven Rosenberg, Wei Xu

Natural gradients have been widely studied from both theoretical and empirical perspectives, and it is commonly believed that natural gradients have advantages over standard (Eucli…