565 citations · 1.2k across the 15 of their papers we have counts for
4 papers · 1 filter
Distillation Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized Neural Network
Bin Dong, Jikai Hou, Yiping Lu +1
Distillation is a method to transfer knowledge from one model to another and often achieves higher accuracy with the same capacity. In this paper, we aim to provide a theoretical u…
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
Yiping Lu, Zhuohan Li, Di He +5
The Transformer architecture is widely used in natural language processing. Despite its success, the design principle of the Transformer remains elusive. In this paper, we provide…
You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu +2
Deep learning achieves state-of-the-art results in many tasks in computer vision and natural language processing. However, recent works have shown that deep networks can be vulnera…
CURE: Curvature Regularization For Missing Data Recovery
Bin Dong, Haocheng Ju, Yiping Lu +1
Missing data recovery is an important and yet challenging problem in imaging and data science. Successful models often adopt certain carefully chosen regularization. Recently, the…