6 papers
PowerCLIP: Powerset Alignment for Contrastive Pre-Training
Masaki Kawamura, Nakamasa Inoue, Rintaro Yanagi +2
Contrastive vision-language pre-training frameworks such as CLIP have demonstrated impressive zero-shot performance across a range of vision-language tasks. Recent studies have sho…
Masked Gated Linear Unit
Yukito Tajima, Nakamasa Inoue, Yusuke Sekikawa +2
Gated Linear Units (GLUs) have become essential components in the feed-forward networks of state-of-the-art Large Language Models (LLMs). However, they require twice as many memory…
On the Relationship Between Double Descent of CNNs and Shape/Texture Bias Under Learning Process
Shun Iwase, Shuya Takahashi, Nakamasa Inoue +3
The double descent phenomenon, which deviates from the traditional bias-variance trade-off theory, attracts considerable research attention; however, the mechanism of its occurrenc…
Formula-Supervised Visual-Geometric Pre-training
Ryosuke Yamada, Kensho Hara, Hirokatsu Kataoka +4
Throughout the history of computer vision, while research has explored the integration of images (visual) and point clouds (geometric), many advancements in image and 3D object rec…
Rethinking Image Super-Resolution from Training Data Perspectives
Go Ohtani, Ryu Tadokoro, Ryosuke Yamada +7
In this work, we investigate the understudied effect of the training data used for image super-resolution (SR). Most commonly, novel SR methods are developed and benchmarked on com…
Scaling Backwards: Minimal Synthetic Pre-training?
Ryo Nakamura, Ryu Tadokoro, Ryosuke Yamada +6
Pre-training and transfer learning are an important building block of current computer vision systems. While pre-training is usually performed on large real-world image datasets, i…