Positional Description Matters for Transformers Arithmetic

Ruoqi Shen, Sébastien Bubeck, Ronen Eldan +3

Transformers, central to the successes in modern Natural Language Processing, often falter on arithmetic tasks despite their vast capabilities --which paradoxically include remarka…