597 citations · 1.4k across the 38 of their papers we have counts for
8 papers · 2 filters
Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective
Bohang Zhang, Du Jiang, Di He +1
Designing neural networks with bounded Lipschitz constant is a promising way to obtain certifiably robust classifiers against adversarial examples. However, the relevant progress f…
One Transformer Can Understand Both 2D & 3D Molecular Data
Shengjie Luo, Tianlang Chen, Yixian Xu +4
Unlike vision and language data which usually has a unique format, molecules can naturally be characterized using different chemical formulations. One can view a molecule as a 2D g…
Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks
Huishuai Zhang, Da Yu, Yiping Lu +1
Adversarial examples, which are usually generated for specific inputs with a specific model, are ubiquitous for neural networks. In this paper we unveil a surprising property of ad…
Is Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?
Chuwei Wang, Shanda Li, Di He +1
The Physics-Informed Neural Network (PINN) approach is a new and promising way to solve partial differential equations using deep learning. The Physics-Informed Loss is the d…
Your Transformer May Not be as Powerful as You Expect
Shengjie Luo, Shanda Li, Shuxin Zheng +3
Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far…
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
Payal Bajaj, Chenyan Xiong, Guolin Ke +7
We present an efficient method of pretraining large-scale autoencoding language models using training signals generated by an auxiliary model. Originated in ELECTRA, this training…