Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning

Md Rifat Arefin, Gopeshh Subbaraj, Nicolas Gontier +4

Decoder-only Transformers often struggle with complex reasoning tasks, particularly arithmetic reasoning requiring multiple sequential operations. In this work, we identify represe…