7 papers
FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback
Xueqing Wu, Zihan Xue, Da Yin +4
We present FronTalk, a benchmark for front-end code generation that pioneers the study of a unique interaction dynamic: conversational code generation with multi-modal feedback. In…
On Asymmetric Optimization of Reasoning and Perception in Vision-Language Model Post-Training
Xueqing Wu, Yu-Chi Lin, Kai-Wei Chang +1
Post-training has greatly improved reasoning in frontier vision-language models, yet its gains for perception remain comparatively limited, creating a bottleneck for end-to-end vis…
LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training
Yiming Wang, Da Yin, Yuedong Cui +8
Digital agents require diverse, large-scale UI trajectories to generalize across real-world tasks, yet collecting such data is prohibitively expensive in both human annotation, inf…
Re-ReST: Reflection-Reinforced Self-Training for Language Agents
Zi-Yi Dou, Cheng-Fu Yang, Xueqing Wu +2
Finetuning language agents with reasoning-action trajectories is effective, but obtaining these trajectories from human annotations or stronger models is costly and sometimes impra…
VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning
Xueqing Wu, Yuheng Ding, Bingxuan Li +4
The ability of large vision-language models (LVLMs) to critique and correct their reasoning is an essential building block towards their self-improvement. However, a systematic ana…
DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation
Xueqing Wu, Rui Zheng, Jingzhen Sha +6
Data analysis is a crucial analytical process to generate in-depth studies and conclusive insights to comprehensively answer a given user query for tabular data. In this work, we a…