9 papers
Fine-Grained Activation Steering: Steering Less, Achieving More
Zijian Feng, Tianjiao Li, Zixiao Zhu +7
Activation steering has emerged as a cost-effective paradigm for modifying large language model (LLM) behaviors. Existing methods typically intervene at the block level, steering t…
Restoring Pruned Large Language Models via Lost Component Compensation
Zijian Feng, Hanzhang Zhou, Zixiao Zhu +5
Pruning is a widely used technique to reduce the size and inference cost of large language models (LLMs), but it often causes performance degradation. To mitigate this, existing re…
Domain Lexical Knowledge-based Word Embedding Learning for Text Classification under Small Data
Zixiao Zhu, Kezhi Mao
Pre-trained language models such as BERT have been proved to be powerful in many natural language processing tasks. But in some text classification applications such as emotion rec…
Rethinking Prompt Optimizers: From Prompt Merits to Optimization
Zixiao Zhu, Hanzhang Zhou, Zijian Feng +5
Prompt optimization (PO) provides a practical way to improve response quality when users lack the time or expertise to manually craft effective prompts. Existing methods typically…
Beyond the Next Token: Towards Prompt-Robust Zero-Shot Classification via Efficient Multi-Token Prediction
Junlang Qian, Zixiao Zhu, Hanzhang Zhou +3
Zero-shot text classification typically relies on prompt engineering, but the inherent prompt brittleness of large language models undermines its reliability. Minor changes in prom…
FreeCtrl: Constructing Control Centers with Feedforward Layers for Learning-Free Controllable Text Generation
Zijian Feng, Hanzhang Zhou, Zixiao Zhu +1
Controllable text generation (CTG) seeks to craft texts adhering to specific attributes, traditionally employing learning-based techniques such as training, fine-tuning, or prefix-…