6 papers
SciFig: Towards Automating Editable Figure Generation for Scientific Papers
Siyuan Huang, Yifan Zhou, Yutong Gao +8
High-quality methodology figures are central to scientific communication, yet they remain difficult and time-consuming to create. Such figures must distill a method's components an…
IMAGHarmony: Controllable Image Editing with Consistent Object Quantity and Layout
Fei Shen, Yutong Gao, Jian Yu +2
Despite advances in diffusion-based image editing, manipulating multi-object scenes remains challenging. Existing approaches often achieve semantic changes at the expense of struct…
CAPT: Confusion-Aware Prompt Tuning for Reducing Vision-Language Misalignment
Maoyuan Shao, Yutong Gao, Xinyang Huang +3
Vision-language models like CLIP have achieved remarkable progress in cross-modal representation learning, yet suffer from systematic misclassifications among visually and semantic…
VideoMiner: Iteratively Grounding Key Frames of Hour-Long Videos via Tree-based Group Relative Policy Optimization
Xinye Cao, Hongcan Guo, Jiawen Qian +6
Understanding hour-long videos with multi-modal large language models (MM-LLMs) enriches the landscape of human-centered AI applications. However, for end-to-end video understandin…
Spotlighter: Revisiting Prompt Tuning from a Representative Mining View
Yutong Gao, Maoyuan Shao, Xinyang Huang +5
CLIP's success has demonstrated that prompt tuning can achieve robust cross-modal semantic alignment for tasks ranging from open-domain recognition to fine-grained classification.…
ContextBLIP: Doubly Contextual Alignment for Contrastive Image Retrieval from Linguistically Complex Descriptions
Honglin Lin, Siyu Li, Guoshun Nan +8
Image retrieval from contextual descriptions (IRCD) aims to identify an image within a set of minimally contrastive candidates based on linguistically complex text. Despite the suc…