Fine-Tuning LLMs with Fine-Grained Human Feedback on Text Spans

Sky CH-Wang, Justin Svegliato, Helen Appel +1

We present a method and dataset for fine-tuning language models with preference supervision using feedback-driven improvement chains. Given a model response, an annotator provides…