30 citations · 78 across the 17 of their papers we have counts for
10 papers · 1 filter
AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP
Joan Nwatu, Tsedeniya Solomon Amare, Longju Bai +17
Understanding which countries are represented in NLP datasets is essential for identifying gaps, targeting data collection, measuring progress, and informing AI policy. However, ge…
A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents
Lingbo Mo, Zeyi Liao, Boyuan Zheng +3
Language agents powered by large language models (LLMs) have seen exploding development. Their capability of using language as a vehicle for thought and communication lends an incr…
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts
Lingfeng Shen, Weiting Tan, Sihao Chen +6
As the influence of large language models (LLMs) spans across global communities, their safety challenges in multilingual settings become paramount for alignment research. This pap…
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Xiang Yue, Yuansheng Ni, Kai Zhang +19
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU…
Mind2Web: Towards a Generalist Agent for the Web
Xiang Deng, Yu Gu, Boyuan Zheng +5
We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website.…
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency
Lingfeng Shen, Weiting Tan, Boyuan Zheng +1
With growing capabilities of large language models, prompting them has become the dominant way to access them. This has motivated the development of strategies for automatically se…
An Empirical Study on Finding Spans
Weiwei Gu, Boyuan Zheng, Yunmo Chen +2
We present an empirical study on methods for span finding, the selection of consecutive tokens in text for some downstream tasks. We focus on approaches that can be employed in tra…
Multilingual Coreference Resolution in Multiparty Dialogue
Boyuan Zheng, Patrick Xia, Mahsa Yarmohammadi +1
Existing multiparty dialogue datasets for entity coreference resolution are nascent, and many challenges are still unaddressed. We create a large-scale dataset, Multilingual Multip…
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning
Cunxiang Wang, Boyuan Zheng, Yuchen Niu +1
To quantitatively and intuitively explore the generalization ability of pre-trained language models (PLMs), we have designed several tasks of arithmetic and logical reasoning. We b…
SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning
Boyuan Zheng, Xiaoyu Yang, Yu-Ping Ruan +4
This paper introduces the SemEval-2021 shared task 4: Reading Comprehension of Abstract Meaning (ReCAM). This shared task is designed to help evaluate the ability of machines in re…