23 citations · 58 across the 7 of their papers we have counts for
7 papers
Bridging Discrete and Backpropagation: Straight-Through and Beyond
Liyuan Liu, Chengyu Dong, Xiaodong Liu +2
Backpropagation, the cornerstone of deep learning, is limited to computing gradients for continuous variables. This limitation poses challenges for problems involving discrete late…
Toward Student-Oriented Teacher Network Training For Knowledge Distillation
Chengyu Dong, Liyuan Liu, Jingbo Shang
How to conduct teacher training for knowledge distillation is still an open problem. It has been widely observed that a best-performing teacher does not necessarily yield the best-…
Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting
Chengyu Dong, Liyuan Liu, Jingbo Shang
We show that label noise exists in adversarial training. Such label noise is due to the mismatch between the true label distribution of adversarial examples and the label inherited…
Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training
Tao Chen, Haochen Shi, Liyuan Liu +4
With recent advances in distantly supervised (DS) relation extraction (RE), considerable attention is attracted to leverage multi-instance learning (MIL) to distill high-quality su…
Multi-head or Single-head? An Empirical Comparison for Transformer Training
Liyuan Liu, Jialu Liu, Jiawei Han
Multi-head attention plays a crucial role in the recent success of Transformer models, which leads to consistent performance improvements over conventional attention in various app…
UCPhrase: Unsupervised Context-aware Quality Phrase Tagging
Xiaotao Gu, Zihan Wang, Zhenyu Bi +4
Identifying and understanding quality phrases from context is a fundamental task in text mining. The most challenging part of this task arguably lies in uncommon, emerging, and dom…
Data Quality Matters For Adversarial Training: An Empirical Study
Chengyu Dong, Liyuan Liu, Jingbo Shang
Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off. These problems pose g…