207 citations · 786 across the 32 of their papers we have counts for
4 papers · 1 filter
Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees
Jiangang Bai, Yujing Wang, Yiren Chen +4
Pre-trained language models like BERT achieve superior performances in various NLP tasks without explicit consideration of syntactic information. Meanwhile, syntactic information h…
AutoADR: Automatic Model Design for Ad Relevance
Yiren Chen, Yaming Yang, Hong Sun +7
Large-scale pre-trained models have attracted extensive attention in the research community and shown promising results on various tasks of natural language processing. However, th…
LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression
Yihuan Mao, Yujing Wang, Chufan Wu +6
BERT is a cutting-edge language representation model pre-trained by a large corpus, which achieves superior performances on various natural language understanding tasks. However, a…
Improving BERT with Self-Supervised Attention
Yiren Chen, Xiaoyu Kou, Jiangang Bai +1
One of the most popular paradigms of applying large pre-trained NLP models such as BERT is to fine-tune it on a smaller dataset. However, one challenge remains as the fine-tuned mo…