1 citations · 2 across the 3 of their papers we have counts for
3 papers
Pre-training Vision Transformers with Very Limited Synthesized Images
Ryo Nakamura, Hirokatsu Kataoka, Sora Takashima +3
Formula-driven supervised learning (FDSL) is a pre-training method that relies on synthetic images generated from mathematical formulae such as fractals. Prior work on FDSL has sho…
Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves
Sora Takashima, Ryo Hayamizu, Nakamasa Inoue +2
Formula-driven supervised learning (FDSL) has been shown to be an effective method for pre-training vision transformers, where ExFractalDB-21k was shown to exceed the pre-training…
Pre-training Vision Transformers with Formula-driven Supervised Learning
Hirokatsu Kataoka, Sora Takashima, Ryo Hayamizu +6
In the present work, we show that the performance of formula-driven supervised learning (FDSL) can match or even exceed that of ImageNet-21k and can approach that of the JFT-300M d…