12 papers · 1 filter
Easier to Judge than to Find: Predicting In-Context Learning Success for Demonstration Selection
Haochun Wang, Chaofen Yang, Jiatong Liu +5
In-context learning (ICL) is highly sensitive to which demonstrations appear in the prompt, but selecting them is expensive because the space of possible demonstration contexts and…
Uncovering the Role of Initial Saliency in U-Shaped Attention Bias: Scaling Initial Token Weight for Enhanced Long-Text Processing
Zewen Qiang, Sendong Zhao, Haochun Wang +2
Large language models (LLMs) have demonstrated strong performance on a variety of natural language processing (NLP) tasks. However, they often struggle with long-text sequences due…
MoGU V2: Toward a Higher Pareto Frontier Between Model Usability and Security
Yanrui Du, Fenglei Fan, Sendong Zhao +3
As Large Language Models (LLMs) increasingly permeate human life, their security has emerged as a critical concern, particularly their ability to maintain harmless responses to mal…
Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint
Yanrui Du, Fenglei Fan, Sendong Zhao +6
Instruction Fine-Tuning (IFT) has been widely adopted as an effective post-training strategy to enhance various abilities of Large Language Models (LLMs). However, prior studies ha…
Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning
Yanrui Du, Sendong Zhao, Jiawei Cao +6
Instruction fine-tuning has emerged as a critical technique for customizing Large Language Models (LLMs) to specific applications. However, recent studies have highlighted signific…
LLMs May Perform MCQA by Selecting the Least Incorrect Option
Haochun Wang, Sendong Zhao, Zewen Qiang +3
In the field of NLP, Large Language Models (LLMs) have markedly enhanced performance across a variety of tasks. However, the comprehensive evaluation of LLMs remains an inevitable…