60 citations · 64 across the 8 of their papers we have counts for
4 papers · 1 filter
CapBencher: Give Your LLM Benchmark a Built-in Alarm for Test-Set Overfitting
Takashi Ishida, Thanawat Lodkaew, Ikko Yamane
Publishing a large language model (LLM) benchmark (especially its ground-truth answers) on the Internet risks contaminating future LLMs and enabling evaluation gaming: it may be un…
Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification
Takashi Ishida, Ikko Yamane, Nontawat Charoenphakdee +2
There is a fundamental limitation in the prediction performance that a machine learning model can achieve due to the inevitable uncertainty of the prediction target. In classificat…
A One-step Approach to Covariate Shift Adaptation
Tianyi Zhang, Ikko Yamane, Nan Lu +1
A default assumption in many machine learning scenarios is that the training and test samples are drawn from the same probability distribution. However, such an assumption is often…
Do We Need Zero Training Loss After Achieving Zero Training Error?
Takashi Ishida, Ikko Yamane, Tomoya Sakai +2
Overparameterized deep networks have the capacity to memorize training data with zero \emph{training error}. Even after memorization, the \emph{training loss} continues to approach…