FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference

Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie +4

Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model that sets the state-of-the-art on many knowledge-intensive NLP tasks. However, the architecture used for Fi…