371 citations · 3.9k across the 110 of their papers we have counts for
25 papers · 1 filter
Wasserstein Contrastive Representation Distillation
Liqun Chen, Dong Wang, Zhe Gan +3
The primary goal of knowledge distillation (KD) is to encapsulate the information of a model learned from a teacher network into a student network, with the latter being more compa…
A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
Linjie Li, Zhe Gan, Jingjing Liu
Large-scale pre-trained multimodal transformers, such as ViLBERT and UNITER, have propelled the state of the art in vision-and-language (V+L) research to a new level. Although achi…
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models
Dong Wang, Yuewei Yang, Chenyang Tao +5
Deep neural networks excel at comprehending complex visual signals, delivering on par or even superior performance to that of human experts. However, ad-hoc visual explanations of…
Cross-Thought for Sentence Encoder Pre-training
Shuohang Wang, Yuwei Fang, Siqi Sun +4
In this paper, we propose Cross-Thought, a novel approach to pre-training sequence encoder, which is instrumental in building reusable sequence embeddings for large-scale NLP tasks…
Multi-Fact Correction in Abstractive Text Summarization
Yue Dong, Shuohang Wang, Zhe Gan +3
Pre-trained neural abstractive summarization systems have dominated extractive strategies on news summarization performance, at least in terms of ROUGE. However, system-generated a…
MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial Network
Yi Wei, Zhe Gan, Wenbo Li +5
We present Mask-guided Generative Adversarial Network (MagGAN) for high-resolution face attribute editing, in which semantic facial masks from a pre-trained face parser are used to…
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
Boxin Wang, Shuohang Wang, Yu Cheng +4
Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies, however, show that such BERT-based models are…
Efficient Robust Training via Backward Smoothing
Jinghui Chen, Yu Cheng, Zhe Gan +2
Adversarial training is so far the most effective strategy in defending against adversarial examples. However, it suffers from high computational costs due to the iterative adversa…
Contrastive Distillation on Intermediate Representations for Language Model Compression
Siqi Sun, Zhe Gan, Yu Cheng +3
Existing language model compression methods mostly use a simple L2 loss to distill knowledge in the intermediate representations of a large BERT model to a smaller one. Although wi…
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
Shuohang Wang, Luowei Zhou, Zhe Gan +5
Transformer has become ubiquitous in the deep learning field. One of the key ingredients that destined its success is the self-attention mechanism, which allows fully-connected con…
Accelerating Real-Time Question Answering via Question Generation
Yuwei Fang, Shuohang Wang, Zhe Gan +3
Although deep neural networks have achieved tremendous success for question answering (QA), they are still suffering from heavy computational and energy cost for real product deplo…
FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding
Yuwei Fang, Shuohang Wang, Zhe Gan +2
Large-scale cross-lingual language models (LM), such as mBERT, Unicoder and XLM, have achieved great success in cross-lingual representation learning. However, when applied to zero…
Graph Optimal Transport for Cross-Domain Alignment
Liqun Chen, Zhe Gan, Yu Cheng +3
Cross-domain alignment between two sets of entities (e.g., objects in an image, words in a sentence) is fundamental to both computer vision and natural language processing. Existin…
CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information
Pengyu Cheng, Weituo Hao, Shuyang Dai +3
Mutual information (MI) minimization has gained considerable interests in various machine learning tasks. However, estimating and minimizing MI in high-dimensional spaces remains a…
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients
Chen Zhu, Yu Cheng, Zhe Gan +3
Adaptive gradient methods such as RMSProp and Adam use exponential moving estimate of the squared gradient to compute adaptive step sizes, achieving better convergence than SGD in…
Large-Scale Adversarial Training for Vision-and-Language Representation Learning
Zhe Gan, Yen-Chun Chen, Linjie Li +3
We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning. VILLA consists of two training stages: (i) task-…
Improving Adversarial Text Generation by Modeling the Distant Future
Ruiyi Zhang, Changyou Chen, Zhe Gan +5
Auto-regressive text generation models usually focus on local fluency, and may cause inconsistent semantic meaning in long text generation. Further, automatically generating words…
Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models
Jize Cao, Zhe Gan, Yu Cheng +3
Recent Transformer-based large-scale pre-trained models have revolutionized vision-and-language (V+L) research. Models such as ViLBERT, LXMERT and UNITER have significantly lifted…
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training
Yizhe Zhang, Guoyin Wang, Chunyuan Li +3
Large-scale pre-trained language models, such as BERT and GPT-2, have achieved excellent performance in language representation learning and free-form text generation. However, the…
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
Linjie Li, Yen-Chun Chen, Yu Cheng +3
We present HERO, a novel framework for large-scale video+language omni-representation learning. HERO encodes multimodal inputs in a hierarchical structure, where local context of a…
APo-VAE: Text Generation in Hyperbolic Space
Shuyang Dai, Zhe Gan, Yu Cheng +3
Natural language often exhibits inherent hierarchical structure ingrained with complex syntax and semantics. However, most state-of-the-art deep generative models learn embeddings…
Contextual Text Style Transfer
Yu Cheng, Zhe Gan, Yizhe Zhang +3
We introduce a new task, Contextual Text Style Transfer - translating a sentence into a desired style with its surrounding context taken into account. This brings two key challenge…
BachGAN: High-Resolution Image Synthesis from Salient Object Layout
Yandong Li, Yu Cheng, Zhe Gan +3
We propose a new task towards more practical application for image generation - high-quality image synthesis from salient object layout. This new setting allows users to provide th…
VIOLIN: A Large-Scale Dataset for Video-and-Language Inference
Jingzhou Liu, Wenhu Chen, Yu Cheng +4
We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text. Given a video clip with aligned subtitles as premise, paired with a nat…