Pre-training Vision Transformers with Very Limited Synthesized Images

Ryo Nakamura, Hirokatsu Kataoka, Sora Takashima +3

Formula-driven supervised learning (FDSL) is a pre-training method that relies on synthetic images generated from mathematical formulae such as fractals. Prior work on FDSL has sho…