Alignment-Aware Model Adaptation via Feedback-Guided Optimization

Gaurav Bhatt, Aditya Chinchure, Jiawei Zhou +1

Fine-tuning is the primary mechanism for adapting foundation models to downstream tasks; however, standard approaches largely optimize task objectives in isolation and do not accou…