4 papers
Attention-Only White-Box Transformer via LeJEPA-Based Self-Supervised Pretraining
Yang Bai, Linyuan Wang, Haoyang Jiang +3
Existing studies on self-supervised learning for white-box networks typically decouple the derivation of white-box networks via optimization algorithms from self-supervised learnin…
A one-step generation model with a Single-Layer Transformer: Layer number re-distillation of FreeFlow
Haonan Wei, Linyuan Wang, Nuolin Sun +3
Currently, Flow matching methods aim to compress the iterative generation process of diffusion models into a few or even a single step, with MeanFlow and FreeFlow being representat…
MFI-ResNet: Efficient ResNet Architecture Optimization via MeanFlow Compression and Selective Incubation
Nuolin Sun, Linyuan Wang, Haonan Wei +2
ResNet has achieved tremendous success in computer vision through its residual connection mechanism. ResNet can be viewed as a discretized form of ordinary differential equations (…
Are classical deep neural networks weakly adversarially robust?
Nuolin Sun, Linyuan Wang, Dongyang Li +2
Adversarial attacks have received increasing attention and it has been widely recognized that classical DNNs have weak adversarial robustness. The most commonly used adversarial de…