CODEBLOCK: Learning to Supervise Code at the Right Granularity

Zhijie Deng, Ling Li, Jinlong Pang +4

Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signa…