178 citations · 347 across the 12 of their papers we have counts for
7 papers · 1 filter
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models
Lei Li, Yankai Lin, Xuancheng Ren +4
Investigating better ways to reuse the released pre-trained language models (PLMs) can significantly reduce the computational cost and the potential environmental side-effects. Thi…
Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models
Lei Li, Yankai Lin, Xuancheng Ren +4
As many fine-tuned pre-trained language models~(PLMs) with promising performance are generously released, investigating better ways to reuse these models is vital as it can greatly…
Learning Relation Alignment for Calibrated Cross-modal Retrieval
Shuhuai Ren, Junyang Lin, Guangxiang Zhao +5
Despite the achievements of large-scale multimodal pre-training approaches, cross-modal retrieval, e.g., image-text retrieval, remains a challenging task. To bridge the semantic ga…
Rethinking and Improving Natural Language Generation with Layer-Wise Multi-View Decoding
Fenglin Liu, Xuancheng Ren, Guangxiang Zhao +4
In sequence-to-sequence learning, e.g., natural language generation, the decoder relies on the attention mechanism to efficiently extract information from the encoder. While it is…
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang +3
Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks. Self-attention is able to model long-term depe…
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
Guangxiang Zhao, Xu Sun, Jingjing Xu +2
In sequence to sequence learning, the self-attention mechanism proves to be highly effective, and achieves significant improvements in many tasks. However, the self-attention mecha…
Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations
Guangxiang Zhao, Jingjing Xu, Qi Zeng +1
This paper explores a new natural language processing task, review-driven multi-label music style classification. This task requires the system to identify multiple styles of music…