Transformers are Stateless Differentiable Neural Computers

Bo Tang, Weiwei Xie

Differentiable Neural Computers (DNCs) were introduced as recurrent architectures equipped with an addressable external memory supporting differentiable read and write operations.…