30 citations · 78 across the 17 of their papers we have counts for
19 papers
AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP
Joan Nwatu, Tsedeniya Solomon Amare, Longju Bai +17
Understanding which countries are represented in NLP datasets is essential for identifying gaps, targeting data collection, measuring progress, and informing AI policy. However, ge…
OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
Mengqi Yuan, Zilong Zhou, Xinzhuang Xiong +33
Existing computer-use benchmarks fail to capture the realism, complexity, and long-horizon demands of real-world computer use, limiting their ability to reveal the limitations of f…
WebGuard: Building a Generalizable Guardrail for Web Agents
Boyuan Zheng, Zeyi Liao, Scott Salisbury +8
The rapid development of autonomous web agents powered by Large Language Models (LLMs), while greatly elevating efficiency, exposes the frontier risk of taking unintended or harmfu…
Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
Boyu Gou, Zanming Huang, Yuting Ning +23
Agentic search such as Deep Research systems-where agents autonomously browse the web, synthesize information, and return comprehensive citation-backed answers-represents a major s…
SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills
Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin +8
To survive and thrive in complex environments, humans have evolved sophisticated self-improvement mechanisms through environment exploration, hierarchical abstraction of experience…
Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents
Yu Gu, Kai Zhang, Yuting Ning +9
Language agents based on large language models (LLMs) have demonstrated great promise in automating web-based tasks. Recent work has shown that incorporating advanced planning algo…
Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents
Boyu Gou, Ruohan Wang, Boyuan Zheng +5
Multimodal large language models (MLLMs) are transforming the capabilities of graphical user interface (GUI) agents, facilitating their transition from controlled simulations to co…
Dual-View Visual Contextualization for Web Navigation
Jihyung Kil, Chan Hee Song, Boyuan Zheng +3
Automatic web navigation aims to build a web agent that can follow language instructions to execute complex and diverse tasks on real-world websites. Existing work primarily takes…
A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents
Lingbo Mo, Zeyi Liao, Boyuan Zheng +3
Language agents powered by large language models (LLMs) have seen exploding development. Their capability of using language as a vehicle for thought and communication lends an incr…
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts
Lingfeng Shen, Weiting Tan, Sihao Chen +6
As the influence of large language models (LLMs) spans across global communities, their safety challenges in multilingual settings become paramount for alignment research. This pap…
GPT-4V(ision) is a Generalist Web Agent, if Grounded
Boyuan Zheng, Boyu Gou, Jihyung Kil +2
The recent development on large multimodal models (LMMs), especially GPT-4V(ision) and Gemini, has been quickly expanding the capability boundaries of multimodal models beyond trad…
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Xiang Yue, Yuansheng Ni, Kai Zhang +19
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU…
Mind2Web: Towards a Generalist Agent for the Web
Xiang Deng, Yu Gu, Boyuan Zheng +5
We introduce Mind2Web, the first dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website.…
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency
Lingfeng Shen, Weiting Tan, Boyuan Zheng +1
With growing capabilities of large language models, prompting them has become the dominant way to access them. This has motivated the development of strategies for automatically se…
An Empirical Study on Finding Spans
Weiwei Gu, Boyuan Zheng, Yunmo Chen +2
We present an empirical study on methods for span finding, the selection of consecutive tokens in text for some downstream tasks. We focus on approaches that can be employed in tra…
Multilingual Coreference Resolution in Multiparty Dialogue
Boyuan Zheng, Patrick Xia, Mahsa Yarmohammadi +1
Existing multiparty dialogue datasets for entity coreference resolution are nascent, and many challenges are still unaddressed. We create a large-scale dataset, Multilingual Multip…
Learn To Remember: Transformer with Recurrent Memory for Document-Level Machine Translation
Yukun Feng, Feng Li, Ziang Song +2
The Transformer architecture has led to significant gains in machine translation. However, most studies focus on only sentence-level translation without considering the context dep…
Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning
Cunxiang Wang, Boyuan Zheng, Yuchen Niu +1
To quantitatively and intuitively explore the generalization ability of pre-trained language models (PLMs), we have designed several tasks of arithmetic and logical reasoning. We b…
SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning
Boyuan Zheng, Xiaoyu Yang, Yu-Ping Ruan +4
This paper introduces the SemEval-2021 shared task 4: Reading Comprehension of Abstract Meaning (ReCAM). This shared task is designed to help evaluate the ability of machines in re…