4 papers · 1 filter
PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts
Anjir Ahmed Chowdhury, Syed Zawad, Xiaolong Ma +2
Parameter-Efficient Fine-Tuning (PEFT) is widely used for adapting Large Language Models (LLMs) for various tasks. Recently, there has been an increasing demand for fine-tuning a s…
When Data is the Algorithm: A Systematic Study and Curation of Preference Optimization Datasets
Aladin Djuhera, Farhan Ahmed, Swanand Ravindra Kadhe +3
Aligning large language models (LLMs) is a central objective of post-training, often achieved through reward modeling and reinforcement learning methods. Among these, direct prefer…
Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance
Aladin Djuhera, Swanand Ravindra Kadhe, Syed Zawad +3
Recent work on large language models (LLMs) has increasingly focused on post-training and alignment with datasets curated to enhance instruction following, world knowledge, and spe…
SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
Aladin Djuhera, Swanand Ravindra Kadhe, Farhan Ahmed +2
Fine-tuning large language models (LLMs) is a common practice to adapt generalist models to specialized domains. However, recent studies show that fine-tuning can erode safety alig…