33 citations · 71 across the 15 of their papers we have counts for
8 papers · 1 filter
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models
Bowen Shen, Zheng Lin, Yuanxin Liu +3
Transformer-based pre-trained language models (PLMs) mostly suffer from excessive overhead despite their advanced capacity. For resource-constrained devices, there is an urgent nee…
A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models
Yuanxin Liu, Fandong Meng, Zheng Lin +5
Despite the remarkable success of pre-trained language models (PLMs), they still face two challenges: First, large-scale PLMs are inefficient in terms of memory footprint and compu…
Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training
Yuanxin Liu, Fandong Meng, Zheng Lin +4
Recent studies on the lottery ticket hypothesis (LTH) show that pre-trained language models (PLMs) like BERT contain matching subnetworks that have similar transfer learning perfor…
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation
Yuanxin Liu, Fandong Meng, Zheng Lin +2
Recently, knowledge distillation (KD) has shown great success in BERT compression. Instead of only learning from the teacher's soft label as in conventional KD, researchers find th…
ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques
Yuanxin Liu, Zheng Lin, Fengcheng Yuan
Pre-trained language models of the BERT family have defined the state-of-the-arts in a wide range of NLP tasks. However, the performance of BERT-based models is mainly driven by th…
Unsupervised Pre-training for Natural Language Generation: A Literature Review
Yuanxin Liu, Zheng Lin
Recently, unsupervised pre-training is gaining increasing popularity in the realm of computational linguistics, thanks to its surprising success in advancing natural language under…
Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations
Fenglin Liu, Yuanxin Liu, Xuancheng Ren +2
In vision-and-language grounding problems, fine-grained representations of the image are considered to be of paramount importance. Most of the current systems incorporate visual fe…
simNet: Stepwise Image-Topic Merging Network for Generating Detailed and Comprehensive Image Captions
Fenglin Liu, Xuancheng Ren, Yuanxin Liu +2
The encode-decoder framework has shown recent success in image captioning. Visual attention, which is good at detailedness, and semantic attention, which is good at comprehensivene…