6 papers
OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesis
Kanzhi Cheng, Zehao Li, Zheng Ma +11
Mobile agents powered by vision-language models have demonstrated impressive capabilities in automating mobile tasks, with recent leading models achieving a marked performance leap…
CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era
Kanzhi Cheng, Wenpo Song, Jiaxin Fan +7
Image captioning has been a longstanding challenge in vision-language research. With the rise of LLMs, modern Vision-Language Models (VLMs) generate detailed and comprehensive imag…
Rethinking Relation Extraction: Beyond Shortcuts to Generalization with a Debiased Benchmark
Liang He, Yougang Chu, Zhen Wu +3
Benchmarks are crucial for evaluating machine learning algorithm performance, facilitating comparison and identifying superior solutions. However, biases within datasets can lead m…
MixRED: A Mix-lingual Relation Extraction Dataset
Lingxing Kong, Yougang Chu, Zheng Ma +3
Relation extraction is a critical task in the field of natural language processing with numerous real-world applications. Existing research primarily focuses on monolingual relatio…
Cobra Effect in Reference-Free Image Captioning Metrics
Zheng Ma, Changxin Wang, Yawen Ouyang +4
Evaluating the compatibility between textual descriptions and corresponding images represents a core endeavor within multi-modal research. In recent years, a proliferation of refer…
Probing Commonsense Reasoning Capability of Text-to-Image Generative Models via Non-visual Description
Mianzhi Pan, Jianfei Li, Mingyue Yu +4
Commonsense reasoning, the ability to make logical assumptions about daily scenes, is one core intelligence of human beings. In this work, we present a novel task and dataset for e…