Compositional Reasoning with Transformers, RNNs, and Chain of Thought

Gilad Yehudai, Noah Amsel, Joan Bruna

It is well understood that different neural network architectures are suited to different tasks, but is there always a single best architecture for a given task? We compare the exp…