Attend or Perish: Benchmarking Attention in Algorithmic Reasoning

Michal Spiegel, Michal Štefánik, Marek Kadlčík +1

Can transformers learn to perform algorithmic tasks reliably across previously unseen input/output domains? While pre-trained language models show solid accuracy on benchmarks inco…