36 citations · 48 across the 11 of their papers we have counts for
11 papers
From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models
Ling Shi, Xinwei Wu, Xiaohu Zhao +7
While mechanistic interpretability tools like Sparse Autoencoders (SAEs) can uncover meaningful features within Large Language Models (LLMs), a critical gap remains in transforming…
Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMs
Xinwei Wu, Heng Liu, Xiaohu Zhao +6
Large Language Models (LLMs) frequently exhibit strong translation abilities, even without task-specific fine-tuning. However, the internal mechanisms governing this innate capabil…
TaP: A Taxonomy-Guided Framework for Automated and Scalable Preference Data Generation
Renren Jin, Tianhao Shen, Xinwei Wu +9
Conducting supervised and preference fine-tuning of large language models (LLMs) requires high-quality datasets to improve their ability to follow instructions and align with human…
Large Language Model Safety: A Holistic Survey
Dan Shi, Tianhao Shen, Yufei Huang +10
The rapid development and deployment of large language models (LLMs) have introduced a new frontier in artificial intelligence, marked by unprecedented capabilities in natural lang…
IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons
Dan Shi, Renren Jin, Tianhao Shen +3
It is widely acknowledged that large language models (LLMs) encode a vast reservoir of knowledge after being trained on mass data. Recent studies disclose knowledge conflicts in LL…
ConTrans: Weak-to-Strong Alignment Engineering via Concept Transplantation
Weilong Dong, Xinwei Wu, Renren Jin +2
Ensuring large language models (LLM) behave consistently with human goals, values, and intentions is crucial for their safety but yet computationally expensive. To reduce the compu…
Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?
Shaoyang Xu, Weilong Dong, Zishan Guo +2
Prior research has revealed that certain abstract concepts are linearly represented as directions in the representation space of LLMs, predominantly centered around English. In thi…
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models
Xinwei Wu, Junzhuo Li, Minghui Xu +4
Large language models pretrained on a huge amount of data capture rich knowledge and information in the training data. The ability of data memorization and regurgitation in pretrai…
Large Language Model Alignment: A Survey
Tianhao Shen, Renren Jin, Yufei Huang +6
Recent years have witnessed remarkable progress made in large language models (LLMs). Such advancements, while garnering significant attention, have concurrently elicited various c…
FewFedWeight: Few-shot Federated Learning Framework across Multiple NLP Tasks
Weilong Dong, Xinwei Wu, Junzhuo Li +3
Massively multi-task learning with large language models has recently made substantial progress on few-shot generalization. However, this is usually performed in a centralized lear…
Swing Distillation: A Privacy-Preserving Knowledge Distillation Framework
Junzhuo Li, Xinwei Wu, Weilong Dong +3
Knowledge distillation (KD) has been widely used for model compression and knowledge transfer. Typically, a big teacher model trained on sufficient data transfers knowledge to a sm…