67 citations · 143 across the 16 of their papers we have counts for
4 papers · 2 filters
WikiWeb2M: A Page-Level Multimodal Wikipedia Dataset
Andrea Burns, Krishna Srinivasan, Joshua Ainslie +5
Webpages have been a rich resource for language and vision-language tasks. Yet only pieces of webpages are kept: image-caption pairs, long text articles, or raw HTML, never all in…
mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences
David Uthus, Santiago Ontañón, Joshua Ainslie +1
We present our work on developing a multilingual, efficient text-to-text transformer that is suitable for handling long inputs. This model, called mLongT5, builds upon the architec…
A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding
Andrea Burns, Krishna Srinivasan, Joshua Ainslie +5
Webpages have been a rich, scalable resource for vision-language and language only tasks. Yet only pieces of webpages are kept in existing datasets: image-caption pairs, long text…
CoLT5: Faster Long-Range Transformers with Conditional Computation
Joshua Ainslie, Tao Lei, Michiel de Jong +9
Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but…