154 citations · 717 across the 63 of their papers we have counts for
64 papers
Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?
Tokio Kajitsuka, Issei Sato
Existing analyses of the expressive capacity of Transformer models have required excessively deep layers for data memorization, leading to a discrepancy with the Transformers actua…
Exploring Weight Balancing on Long-Tailed Recognition Problem
Naoya Hasegawa, Issei Sato
Recognition problems in long-tailed data, in which the sample size per class is heavily skewed, have gained importance because the distribution of the sample size per class in a da…
Excess risk analysis for epistemic uncertainty with application to variational inference
Futoshi Futami, Tomoharu Iwata, Naonori Ueda +2
Bayesian deep learning plays an important role especially for its ability evaluating epistemic uncertainty (EU). Due to computational complexity issues, approximation methods such…
Analyzing Lottery Ticket Hypothesis from PAC-Bayesian Theory Perspective
Keitaro Sakamoto, Issei Sato
The lottery ticket hypothesis (LTH) has attracted attention because it can explain why over-parameterized models often show high generalization ability. It is known that when we us…
Goldilocks-curriculum Domain Randomization and Fractal Perlin Noise with Application to Sim2Real Pneumonia Lesion Detection
Takahiro Suzuki, Shouhei Hanaoka, Issei Sato
A computer-aided detection (CAD) system based on machine learning is expected to assist radiologists in making a diagnosis. It is desirable to build CAD systems for the various typ…
Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey
Kento Nozawa, Issei Sato
Representation learning enables us to automatically extract generic feature representations from a dataset to solve another machine learning task. Recently, extracted feature repre…