178 citations · 496 across the 23 of their papers we have counts for
23 papers
Incorporating Pre-trained Model Prompting in Multimodal Stock Volume Movement Prediction
Ruibo Chen, Zhiyuan Zhang, Yi Liu +3
Multimodal stock trading volume movement prediction with stock-related news is one of the fundamental problems in the financial area. Existing multimodal works that train models fr…
Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias
Zhiyuan Zhang, Deli Chen, Hao Zhou +3
Pre-trained Language Models (PLMs) may be poisonous with backdoors or bias injected by the suspicious attacker during the fine-tuning process. A core challenge of purifying potenti…
Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models
Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma +2
Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Processing (NLP), DNNs are often backdoored during the fine-tuning process of a larg…
Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks
Sishuo Chen, Wenkai Yang, Zhiyuan Zhang +2
Natural language processing (NLP) models are known to be vulnerable to backdoor attacks, which poses a newly arisen threat to NLP models. Prior online backdoor defense methods for…
GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization
Zhiyuan Zhang, Ruixuan Luo, Qi Su +1
Recently, Sharpness-Aware Minimization (SAM) algorithm has shown state-of-the-art generalization abilities in vision tasks. It demonstrates that flat minima tend to imply better ge…
Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation
Zhiyuan Zhang, Qi Su, Xu Sun
Despite the potential of federated learning, it is known to be vulnerable to backdoor attacks. Many robust federated aggregation methods are proposed to reduce the potential backdo…
Stock Trading Volume Prediction with Dual-Process Meta-Learning
Ruibo Chen, Wei Li, Zhiyuan Zhang +3
Volume prediction is one of the fundamental objectives in the Fintech area, which is helpful for many downstream tasks, e.g., algorithmic trading. Previous methods mostly learn a u…
Distributional Correlation--Aware Knowledge Distillation for Stock Trading Volume Prediction
Lei Li, Zhiyuan Zhang, Ruihan Bao +2
Traditional knowledge distillation in classification problems transfers the knowledge via class correlations in the soft label produced by teacher models, which are not available i…
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
Runxin Xu, Fuli Luo, Zhiyuan Zhang +4
Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus aris…
Adversarial Parameter Defense by Multi-Step Risk Minimization
Zhiyuan Zhang, Ruixuan Luo, Xuancheng Ren +3
Previous studies demonstrate DNNs' vulnerability to adversarial examples and adversarial training can establish a defense to adversarial examples. In addition, recent studies show…
How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data
Zhiyuan Zhang, Lingjuan Lyu, Weiqiang Wang +2
Since training a large-scale backdoored model from scratch requires a large training dataset, several recent attacks have considered to inject backdoors into a trained clean model…
ASAT: Adaptively Scaled Adversarial Training in Time Series
Zhiyuan Zhang, Wei Li, Ruihan Bao +3
Adversarial training is a method for enhancing neural networks to improve the robustness against adversarial examples. Besides the security concerns of potential adversarial exampl…
Rethinking Skip Connection with Layer Normalization in Transformers and ResNets
Fenglin Liu, Xuancheng Ren, Zhiyuan Zhang +2
Skip connection, is a widely-used technique to improve the performance and the convergence of deep neural networks, which is believed to relieve the difficulty in optimization due…
Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models
Wenkai Yang, Lei Li, Zhiyuan Zhang +3
Recent studies have revealed a security threat to natural language processing (NLP) models, called the Backdoor Attack. Victim models can maintain competitive performance on clean…
Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network
Ruixuan Luo, Wei Li, Zhiyuan Zhang +3
Clustering is one of the fundamental problems in unsupervised learning. Recent deep learning based methods focus on learning clustering oriented representations. Among those method…
Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption
Xu Sun, Zhiyuan Zhang, Xuancheng Ren +2
We argue that the vulnerability of model parameters is of crucial value to the study of model robustness and generalization but little research has been devoted to understanding th…
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang +3
Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks. Self-attention is able to model long-term depe…
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
Guangxiang Zhao, Xu Sun, Jingjing Xu +2
In sequence to sequence learning, the self-attention mechanism proves to be highly effective, and achieves significant improvements in many tasks. However, the self-attention mecha…
Understanding and Improving Layer Normalization
Jingjing Xu, Xu Sun, Zhiyuan Zhang +2
Layer normalization (LayerNorm) is a technique to normalize the distributions of intermediate layers. It enables smoother gradients, faster training, and better generalization accu…
PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
Ruixuan Luo, Jingjing Xu, Yi Zhang +3
Chinese word segmentation (CWS) is a fundamental step of Chinese natural language processing. In this paper, we build a new toolkit, named PKUSEG, for multi-domain word segmentatio…
Memorized Sparse Backpropagation
Zhiyuan Zhang, Pengcheng Yang, Xuancheng Ren +2
Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existi…
Automatic Translating between Ancient Chinese and Contemporary Chinese with Limited Aligned Corpora
Zhiyuan Zhang, Wei Li, Qi Su
The Chinese language has evolved a lot during the long-term development. Therefore, native speakers now have trouble in reading sentences written in ancient Chinese. In this paper,…
Building an Ellipsis-aware Chinese Dependency Treebank for Web Text
Xuancheng Ren, Xu Sun, Ji Wen +3
Web 2.0 has brought with it numerous user-produced data revealing one's thoughts, experiences, and knowledge, which are a great source for many tasks, such as information extractio…