111 citations · 283 across the 44 of their papers we have counts for
10 papers · 1 filter
Knowledge Distillation under Teacher Misspecification: An Order-Parameter Analysis of the Gap between Teacher Mimicry and Task Performance
Kazuyuki Hara, Hideitsu Hino
Knowledge distillation trains a small student model to reproduce the outputs of a large teacher model, and its progress is typically monitored through the teacher--student discrepa…
LIG: Layer-wise Integrated Gradients for Within-Layer Flow Analysis in Transformers
Eight Suzuki, Hideitsu Hino, Noboru Murata
Transformers achieve strong performance, but their internal computations remain opaque. We view each Transformer layer as a dynamic graph whose nodes are token representations and…
Sobolev--Ricci Curvature
Kyoichi Iwasaki, Tam Le, Hideitsu Hino
Ricci curvature is a fundamental concept in differential geometry for encoding local geometric structure, and its graph-based analogues have recently gained prominence as practical…
Generalized Sobolev IPM for Graph-Based Measures
Tam Le, Truyen Nguyen, Hideitsu Hino +1
We study the Sobolev IPM problem for measures supported on a graph metric space, where critic function is constrained to lie within the unit ball defined by Sobolev norm. While Le…
A Short Survey on Importance Weighting for Machine Learning
Masanari Kimura, Hideitsu Hino
Importance weighting is a fundamental procedure in statistics and machine learning that weights the objective function or probability distribution based on the importance of the in…
Information Geometrically Generalized Covariate Shift Adaptation
Masanari Kimura, Hideitsu Hino
Many machine learning methods assume that the training and test data follow the same distribution. However, in the real world, this assumption is very often violated. In particular…
Unsupervised Domain Adaptation for Extra Features in the Target Domain Using Optimal Transport
Toshimitsu Aritake, Hideitsu Hino
Domain adaptation aims to transfer knowledge of labeled instances obtained from a source domain to a target domain to fill the gap between the domains. Most domain adaptation metho…
Active Learning: Problem Settings and Recent Developments
Hideitsu Hino
In supervised learning, acquiring labeled training data for a predictive model can be very costly, but acquiring a large amount of unlabeled data is often quite easy. Active learni…
On a convergence property of a geometrical algorithm for statistical manifolds
Shotaro Akaho, Hideitsu Hino, Noboru Murata
In this paper, we examine a geometrical projection algorithm for statistical inference. The algorithm is based on Pythagorean relation and it is derivative-free as well as represen…
New Probabilistic Bounds on Eigenvalues and Eigenvectors of Random Kernel Matrices
Nima Reyhani, Hideitsu Hino, Ricardo Vigario
Kernel methods are successful approaches for different machine learning problems. This success is mainly rooted in using feature maps and kernel matrices. Some methods rely on the…