145 citations · 193 across the 4 of their papers we have counts for
5 papers
Mephisto: A Framework for Portable, Reproducible, and Iterative Crowdsourcing
Jack Urbanek, Pratik Ringshia
We introduce Mephisto, a framework to make crowdsourcing for research more reproducible, transparent, and collaborative. Mephisto provides abstractions that cover a broad set of ta…
Dynabench: Rethinking Benchmarking in NLP
Douwe Kiela, Max Bartolo, Yixin Nie +16
We introduce Dynabench, an open-source platform for dynamic dataset creation and model benchmarking. Dynabench runs in a web browser and supports human-and-model-in-the-loop datase…
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
Stephen Roller, Y-Lan Boureau, Jason Weston +13
We present our view of what is necessary to build an engaging open-domain conversational agent: covering the qualities of such an agent, the pieces of the puzzle that have been bui…
The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes
Douwe Kiela, Hamed Firooz, Aravind Mohan +4
This work proposes a new challenge set for multimodal classification, focusing on detecting hate speech in multimodal memes. It is constructed such that unimodal models struggle an…
Generating Interactive Worlds with Text
Angela Fan, Jack Urbanek, Pratik Ringshia +8
Procedurally generating cohesive and interesting game environments is challenging and time-consuming. In order for the relationships between the game elements to be natural, common…