PruneCD: Contrasting Pruned Self Model to Improve Decoding Factuality

Byeongho Yu, Changhun Lee, Jungyu Jin +1

To mitigate the hallucination problem in large language models, DoLa exploits early exit logits from the same model as a contrastive prior. However, we found that these early exit…