178 citations · 347 across the 12 of their papers we have counts for
12 papers
When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learning
Wenkai Yang, Yankai Lin, Guangxiang Zhao +3
Federated Learning has become a widely-used framework which allows learning a global model on decentralized local datasets under the condition of protecting local data privacy. How…
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models
Lei Li, Yankai Lin, Xuancheng Ren +4
Investigating better ways to reuse the released pre-trained language models (PLMs) can significantly reduce the computational cost and the potential environmental side-effects. Thi…
Delving into the Openness of CLIP
Shuhuai Ren, Lei Li, Xuancheng Ren +2
Contrastive Language-Image Pre-training (CLIP) formulates image classification as an image-to-text matching task, i.e., matching images to the corresponding natural language descri…
Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models
Lei Li, Yankai Lin, Xuancheng Ren +4
As many fine-tuned pre-trained language models~(PLMs) with promising performance are generously released, investigating better ways to reuse these models is vital as it can greatly…
Topology-Imbalance Learning for Semi-Supervised Node Classification
Deli Chen, Yankai Lin, Guangxiang Zhao +4
The class imbalance problem, as an important issue in learning node representations, has drawn increasing attention from the community. Although the imbalance considered by existin…
Well-classified Examples are Underestimated in Classification with Deep Neural Networks
Guangxiang Zhao, Wenkai Yang, Xuancheng Ren +3
The conventional wisdom behind learning deep classification models is to focus on bad-classified examples and ignore well-classified examples that are far from the decision boundar…
Learning Relation Alignment for Calibrated Cross-modal Retrieval
Shuhuai Ren, Junyang Lin, Guangxiang Zhao +5
Despite the achievements of large-scale multimodal pre-training approaches, cross-modal retrieval, e.g., image-text retrieval, remains a challenging task. To bridge the semantic ga…
Rethinking and Improving Natural Language Generation with Layer-Wise Multi-View Decoding
Fenglin Liu, Xuancheng Ren, Guangxiang Zhao +4
In sequence-to-sequence learning, e.g., natural language generation, the decoder relies on the attention mechanism to efficiently extract information from the encoder. While it is…
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang +3
Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks. Self-attention is able to model long-term depe…
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
Guangxiang Zhao, Xu Sun, Jingjing Xu +2
In sequence to sequence learning, the self-attention mechanism proves to be highly effective, and achieves significant improvements in many tasks. However, the self-attention mecha…
Understanding and Improving Layer Normalization
Jingjing Xu, Xu Sun, Zhiyuan Zhang +2
Layer normalization (LayerNorm) is a technique to normalize the distributions of intermediate layers. It enables smoother gradients, faster training, and better generalization accu…
Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations
Guangxiang Zhao, Jingjing Xu, Qi Zeng +1
This paper explores a new natural language processing task, review-driven multi-label music style classification. This task requires the system to identify multiple styles of music…