Value-Aware Numerical Representations for Transformer Language Models

Andreea Dutulescu, Stefan Ruseti, Mihai Dascalu

Transformer-based language models often achieve strong results on mathematical reasoning benchmarks while remaining fragile on basic numerical understanding and arithmetic operatio…