12 papers
SurgNarrator: A Generative Retrieval Framework for Surgical Video Understanding
Yuqing Feng, Jiawei Ma, Kevin Qinghong Lin +6
Surgical procedures unfold as structured and recurring clinical events, whose real-time understanding via intraoperative surgical videos is critical for intraoperative decision-mak…
Demo2Tutorial: From Human Experience to Multimodal Software Tutorials
Zechen Bai, Zhiheng Chen, Yiqi Lin +5
Human experience in digital environments offers a vast, underexplored resource of authentic, untrimmed interactions that contain rich procedural knowledge. We introduce Demo2Tutori…
FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
Mingyu Ouyang, Kevin Qinghong Lin, Mike Zheng Shou +1
Vision-Language Models (VLMs) have shown remarkable performance in User Interface (UI) grounding tasks, driven by their ability to process increasingly high-resolution screenshots.…
ShowUI-: Flow-based Generative Models as GUI Dexterous Hands
Siyuan Hu, Kevin Qinghong Lin, Mike Zheng Shou
Building intelligent agents capable of dexterous manipulation is essential for achieving human-like automation in both robotics and digital environments. However, existing GUI agen…
Reinforcement Learning for Large Model: A Survey
Weijia Wu, Chen Gao, Joya Chen +6
Recent advances at the intersection of reinforcement learning (RL) and visual intelligence have enabled agents that not only perceive complex visual scenes but also reason, generat…
Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models
Jiaqi Wang, Kevin Qinghong Lin, James Cheng +1
Reinforcement Learning (RL) has proven to be an effective post-training strategy for enhancing reasoning in vision-language models (VLMs). Group Relative Policy Optimization (GRPO)…