8 papers · 1 filter
Why Generate a Click? Plug-in Decision Heads for Web Agents
Chengguang Gan, Yunhao Liang, Hanjun Wei +3
Web agents built on vision language models decide every action by generating text such as click(12), although most of that text encodes a choice among a few action types and a few…
Certified Selective Automation of LLM Agent Evaluation
Chengguang Gan, Yunhao Liang, Qinghao Zhang +1
Evaluating LLM agents still ends with a human reading trajectories, because automatic judges carry no guarantee on how often they are wrong. We ask the operational question: what f…
Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents
Chengguang Gan, Yunhao Liang, QingHao Zhang +1
Web agents are usually evaluated in live environments, where environment state and judge models drift between runs, so the same checkpoint rarely reproduces the same score, making…
How Output Format Confounds Data Quality and Capability in Instruction Tuning
Chengguang Gan, Hanjun Wei, Yunhao Liang +3
Instruction-tuning data are judged by quality metrics, and tuned models are judged by benchmarks, but both judgments pass through an output interface: the surface format in which a…
Joint Training Is Not Enough: Conditioned Cross-Granularity Training for Multimodal Document Understanding
Chengguang Gan, Yunhao Liang, Hanjun Wei +2
The Mutual Reinforcement Effect (MRE) asks whether a fine, span-level and a coarse, document-level task help each other when one model handles both. We test it in multimodal docume…
GuideWeb: A Benchmark for Automatic In-App Guide Generation on Real-World Web UIs
Chengguang Gan, Yoshihiro Tsujii, Yunhao Liang +3
Digital Adoption Platform (DAP) provide web-based overlays that deliver operation guidance and contextual hints to help users navigate complex websites. Although modern DAP tools e…
M-MRE: Extending the Mutual Reinforcement Effect to Multimodal Information Extraction
Chengguang Gan, Zhixi Cai, Yanbin Wei +3
Mutual Reinforcement Effect (MRE) is an emerging subfield at the intersection of information extraction and model interpretability. MRE aims to leverage the mutual understanding be…
A Multilingual Dataset and Empirical Validation for the Mutual Reinforcement Effect in Information Extraction
Chengguang Gan, Sunbowen Lee, Qingyu Yin +9
The Mutual Reinforcement Effect (MRE) describes a phenomenon in information extraction where word-level and sentence-level tasks can mutually improve each other when jointly modele…