13 citations · 34 across the 9 of their papers we have counts for
17 papers
Efficiently Tuned Parameters are Task Embeddings
Wangchunshu Zhou, Canwen Xu, Julian McAuley
Intermediate-task transfer can benefit a wide range of NLP tasks with properly selected source datasets. However, it is computationally infeasible to experiment with all intermedia…
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning
Tiannan Wang, Wangchunshu Zhou, Yan Zeng +1
Pre-trained vision-language models (VLMs) have achieved impressive results in a range of vision-language tasks. However, popular VLMs usually consist of hundreds of millions of par…
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
Wangchunshu Zhou, Yan Zeng, Shizhe Diao +1
Recent advances in vision-language pre-training (VLP) have demonstrated impressive performance in a range of vision-language (VL) tasks. However, there exist several challenges for…
Contextual Representation Learning beyond Masked Language Modeling
Zhiyi Fu, Wangchunshu Zhou, Jingjing Xu +2
How do masked language models (MLMs) such as BERT learn contextual representations? In this work, we analyze the learning dynamics of MLMs. We find that MLMs adopt sampled embeddin…
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
Canwen Xu, Wangchunshu Zhou, Tao Ge +3
Recent studies on compression of pretrained language models (e.g., BERT) usually use preserved accuracy as the metric for evaluation. In this paper, we propose two new metrics, lab…
Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
Wangchunshu Zhou, Qifei Li, Chenle Li
In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. In contras…