most citedAn Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

57 citations · 99 across the 11 of their papers we have counts for

collaborators

11 papers

cs.CL2023★ 57 cited

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

Yun Luo, Zhen Yang, Fandong Meng +3

Catastrophic forgetting (CF) is a phenomenon that occurs in machine learning when a model forgets previously learned information while acquiring new knowledge for achieving a satis…

q-bio.BM2023★ 1 cited

Efficient Prediction of Peptide Self-assembly through Sequential and Graphical Encoding

Zihan Liu, Jiaqi Wang, Yun Luo +3

In recent years, there has been an explosion of research on the application of deep learning to the prediction of various peptide properties, due to the significant development and…

cs.LG2023★ 1 cited

Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion

Yun Luo, Xiaotian Lin, Zhen Yang +3

Task-incremental continual learning refers to continually training a model in a sequence of tasks while overcoming the problem of catastrophic forgetting (CF). The issue arrives fo…

cs.CL2023★ 1 cited

Investigating Forgetting in Pre-Trained Representations Through Continual Learning

Yun Luo, Zhen Yang, Xuefeng Bai +3

Representation forgetting refers to the drift of contextualized representations during continual training. Intuitively, the representation forgetting can influence the general know…

cs.LG2023★ 9 cited

Towards Reasonable Budget Allocation in Untargeted Graph Structure Attacks via Gradient Debias

Zihan Liu, Yun Luo, Lirong Wu +2

It has become cognitive inertia to employ cross-entropy loss function in classification related tasks. In the untargeted attacks on graph structure, the gradients derived from the…

cs.CL2023

Improving (Dis)agreement Detection with Inductive Social Relation Information From Comment-Reply Interactions

Yun Luo, Zihan Liu, Stan Z. Li +1

(Dis)agreement detection aims to identify the authors' attitudes or positions (\textit{agree, disagree, neutral}) towards a specific text. It is limited for existing methods merely…

cs.CR2022★ 3 cited

Are Gradients on Graph Structure Reliable in Gray-box Attacks?

Zihan Liu, Yun Luo, Lirong Wu +3

Graph edge perturbations are dedicated to damaging the prediction of graph neural networks by modifying the graph structure. Previous gray-box attackers employ gradients from the s…

cs.CL2022★ 10 cited

Mere Contrastive Learning for Cross-Domain Sentiment Analysis

Yun Luo, Fang Guo, Zihan Liu +1

Cross-domain sentiment analysis aims to predict the sentiment of texts in the target domain using the model trained on the source domain to cope with the scarcity of labeled data.…

cs.LG2022

What Does the Gradient Tell When Attacking the Graph Structure

Zihan Liu, Ge Wang, Yun Luo +1

Recent research has revealed that Graph Neural Networks (GNNs) are susceptible to adversarial attacks targeting the graph structure. A malicious attacker can manipulate a limited n…

cs.CL2022★ 12 cited

Exploiting Sentiment and Common Sense for Zero-shot Stance Detection

Yun Luo, Zihan Liu, Yuefeng Shi +2

The stance detection task aims to classify the stance toward given documents and topics. Since the topics can be implicit in documents and unseen in training data for zero-shot set…

cs.CL2022★ 5 cited

Challenges for Open-domain Targeted Sentiment Analysis

Yun Luo, Hongjie Cai, Linyi Yang +3

Since previous studies on open-domain targeted sentiment analysis are limited in dataset domain variety and sentence level, we propose a novel dataset consisting of 6,013 human-lab…