5 papers
Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding
Yan Zhang, Daiqing Wu, Huawen Shen +2
Graphical User Interface (GUI) grounding maps natural language instructions to the visual coordinates of target elements and serves as a core capability for autonomous GUI agents.…
VidText: Towards Comprehensive Evaluation for Video Text Understanding
Zhoufaran Yang, Yan Shu, Jing Wang +8
Visual texts embedded in videos carry rich semantic information, which is crucial for both holistic video understanding and fine-grained reasoning about local human actions. Howeve…
When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understanding
Yan Shu, Hangui Lin, Yexin Liu +7
Large Multimodal Models (LMMs) have achieved impressive progress in visual perception and reasoning. However, when confronted with visually ambiguous or non-semantic scene text, th…
Gather and Trace: Rethinking Video TextVQA from an Instance-oriented Perspective
Yan Zhang, Gangyan Zeng, Daiqing Wu +5
Video text-based visual question answering (Video TextVQA) aims to answer questions by explicitly reading and reasoning about the text involved in a video. Most works in this field…
Track the Answer: Extending TextVQA from Image to Video with Spatio-Temporal Clues
Yan Zhang, Gangyan Zeng, Huawen Shen +3
Video text-based visual question answering (Video TextVQA) is a practical task that aims to answer questions by jointly reasoning textual and visual information in a given video. I…