6 papers
CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language Misalignment
Maoyuan Shao, Yutong Gao, Xinyang Huang +3
Vision-language models like CLIP have achieved remarkable progress in cross-modal representation learning, yet suffer from systematic misclassifications among visually and semantic…
SciFig: Towards Automating Editable Figure Generation for Scientific Papers
Siyuan Huang, Yifan Zhou, Yutong Gao +8
High-quality methodology figures are central to scientific communication, yet they remain difficult and time-consuming to create. Such figures must distill a method's components an…
VideoMiner: Iteratively Grounding Key Frames of Hour-Long Videos via Tree-based Group Relative Policy Optimization
Xinye Cao, Hongcan Guo, Jiawen Qian +6
Understanding hour-long videos with multi-modal large language models (MM-LLMs) enriches the landscape of human-centered AI applications. However, for end-to-end video understandin…
Spotlighter: Revisiting Prompt Tuning from a Representative Mining View
Yutong Gao, Maoyuan Shao, Xinyang Huang +5
CLIP's success has demonstrated that prompt tuning can achieve robust cross-modal semantic alignment for tasks ranging from open-domain recognition to fine-grained classification.…
ContextBLIP: Doubly Contextual Alignment for Contrastive Image Retrieval from Linguistically Complex Descriptions
Honglin Lin, Siyu Li, Guoshun Nan +8
Image retrieval from contextual descriptions (IRCD) aims to identify an image within a set of minimally contrastive candidates based on linguistically complex text. Despite the suc…
Clicking Matters:Towards Interactive Human Parsing
Yutong Gao, Liqian Liang, Congyan Lang +3
In this work, we focus on Interactive Human Parsing (IHP), which aims to segment a human image into multiple human body parts with guidance from users' interactions. This new task…