9 papers
Fine-Grained Activation Steering: Steering Less, Achieving More
Zijian Feng, Tianjiao Li, Zixiao Zhu +7
Activation steering has emerged as a cost-effective paradigm for modifying large language model (LLM) behaviors. Existing methods typically intervene at the block level, steering t…
Rethinking Prompt Optimizers: From Prompt Merits to Optimization
Zixiao Zhu, Hanzhang Zhou, Zijian Feng +5
Prompt optimization (PO) provides a practical way to improve response quality when users lack the time or expertise to manually craft effective prompts. Existing methods typically…
Restoring Pruned Large Language Models via Lost Component Compensation
Zijian Feng, Hanzhang Zhou, Zixiao Zhu +5
Pruning is a widely used technique to reduce the size and inference cost of large language models (LLMs), but it often causes performance degradation. To mitigate this, existing re…
Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification via Efficient Multi-Token Prediction
Junlang Qian, Zixiao Zhu, Hanzhang Zhou +3
Zero-shot text classification typically relies on prompt engineering, but the inherent prompt brittleness of large language models undermines its reliability. Minor changes in prom…
UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation
Hanzhang Zhou, Zijian Feng, Zixiao Zhu +2
Large language models (LLMs) have demonstrated impressive capabilities in various tasks using the in-context learning (ICL) paradigm. However, their effectiveness is often compromi…
Logit Separability-Driven Samples and Multiple Class-Related Words Selection for Advancing In-Context Learning
Zhu Zixiao, Feng Zijian, Zhou Hanzhang +2
Effective organization of in-context learning (ICL) demonstrations is key to improving the quality of large language model (LLM) responses. To create better sample-label pairs that…