352 citations · 690 across the 42 of their papers we have counts for
29 papers · 1 filter
Benchmarking Foundation and Large Language Models for Few-Shot Medical Image Segmentation
Jinghong Liu, Yuchuan Deng, Fanping Liu +2
Few-shot medical image segmentation (FS-MIS) aims to segment novel regions of interest (ROIs) from a few annotated support examples. Despite rapid progress, existing FS-MIS solutio…
Stage-adaptive Token Selection for Efficient Omni-modal LLMs
Zijie Xin, Jie Yang, Ruixiang Zhao +4
The unified audio-visual understanding capability of omni-modal large language models (om-LLMs) comes at the high cost of processing tens of thousands of visual and audio tokens th…
OmniPro: A Comprehensive Benchmark for Omni-Proactive Streaming Video Understanding
Ruixiang Zhao, Jie Yang, Zijie Xin +4
Omni-proactive streaming video understanding, i.e., autonomously deciding when to speak and what to say from continuous audio-visual streams, is an emerging capability of omni-moda…
Fundus-R1: Training a Fundus-Reading MLLM on Public Data
Yuchuan Deng, Qijie Wei, Kaiheng Qian +8
Fundus imaging such as CFP, OCT and UWF is crucial for the early detection of retinal anomalies and diseases. Fundus image understanding, due to its knowledge-intensive nature, pos…
Early Intervention for VFM-based Multimodal Medical Image Classification
Qijie Wei, Hailan Lin, Xirong Li
Current methods for multimodal medical image classification (M3IC) face two major challenges. First, the prevailing "fusion after unimodal image embedding" paradigm cannot fully ex…
SAVE: Speech-Aware Video Representation Learning for Video-Text Retrieval
Ruixiang Zhao, Zhihao Xu, Bangxiang Lan +3
For video-text retrieval, the use of CLIP has been a de facto choice. Since CLIP provides only image and text encoders, this consensus has led to a biased paradigm that entirely ig…
Cross-modal Fundus Image Registration under Large FoV Disparity
Hongyang Li, Junyi Tao, Qijie Wei +4
Previous work on cross-modal fundus image registration (CMFIR) assumes small cross-modal Field-of-View (FoV) disparity. By contrast, this paper is targeted at a more challenging sc…
Hybrid-Tower: Fine-grained Pseudo-query Interaction and Generation for Text-to-Video Retrieval
Bangxiang Lan, Ruobing Xie, Ruixiang Zhao +4
The Text-to-Video Retrieval (T2VR) task aims to retrieve unlabeled videos by textual queries with the same semantic meanings. Recent CLIP-based approaches have explored two framewo…
Learning Partially-Decorrelated Common Spaces for Ad-hoc Video Search
Fan Hu, Zijie Xin, Xirong Li
Ad-hoc Video Search (AVS) involves using a textual query to search for multiple relevant videos in a large collection of unlabeled short videos. The main challenge of AVS is the vi…
FunBench: Benchmarking Fundus Reading Skills of MLLMs
Qijie Wei, Kaiheng Qian, Xirong Li
Multimodal Large Language Models (MLLMs) have shown significant potential in medical image analysis. However, their capabilities in interpreting fundus images, a critical skill for…
Multi-Object Sketch Animation by Scene Decomposition and Motion Planning
Jingyu Liu, Zijie Xin, Yuhan Fu +3
Sketch animation, which brings static sketches to life by generating dynamic video sequences, has found widespread applications in GIF design, cartoon production, and daily enterta…
Convolutional Prompting for Broad-Domain Retinal Vessel Segmentation
Qijie Wei, Weihong Yu, Xirong Li
Previous research on retinal vessel segmentation is targeted at a specific image domain, mostly color fundus photography (CFP). In this paper we make a brave attempt to attack a mo…
Beyond Coarse-Grained Matching in Video-Text Retrieval
Aozhu Chen, Hazel Doughty, Xirong Li +1
Video-text retrieval has seen significant advancements, yet the ability of models to discern subtle differences in captions still requires verification. In this paper, we introduce…
D&M: Enriching E-commerce Videos with Sound Effects by Key Moment Detection and SFX Matching
Jingyu Liu, Minquan Wang, Ye Ma +5
Videos showcasing specific products are increasingly important for E-commerce. Key moments naturally exist as the first appearance of a specific product, presentation of its distin…
PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset
Jiazhen Liu, Yuhan Fu, Ruobing Xie +5
Multimodal Large Language Models (MLLMs) hallucinate, resulting in an emerging topic of visual hallucination evaluation (VHE). This paper contributes a ChatGPT-Prompted visual hall…
TeachCLIP: Multi-Grained Teaching for Efficient Text-to-Video Retrieval
Kaibin Tian, Ruixiang Zhao, Hu Hu +4
For text-to-video retrieval (T2VR), which aims to retrieve unlabeled videos by ad-hoc textual queries, CLIP-based methods are dominating. Compared to CLIP4Clip which is efficient a…
Renmin University of China at TRECVID 2022: Improving Video Search by Feature Fusion and Negation Understanding
Xirong Li, Aozhu Chen, Ziyue Wang +4
We summarize our TRECVID 2022 Ad-hoc Video Search (AVS) experiments. Our solution is built with two new techniques, namely Lightweight Attentional Feature Fusion (LAFF) for combini…
PRVR: Partially Relevant Video Retrieval
Xianke Chen, Daizong Liu, Xun Yang +4
In current text-to-video retrieval (T2VR), videos to be retrieved have been properly trimmed so that a correspondence between the videos and ad-hoc textual queries naturally exists…
Semi-Supervised Keypoint Detector and Descriptor for Retinal Image Matching
Jiazhen Liu, Xirong Li, Qijie Wei +2
For retinal image matching (RIM), we propose SuperRetina, the first end-to-end method with jointly trainable keypoint detector and descriptor. SuperRetina is trained in a novel sem…
Lesion Localization in OCT by Semi-Supervised Object Detection
Yue Wu, Yang Zhou, Jianchun Zhao +4
Over 300 million people worldwide are affected by various retinal diseases. By noninvasive Optical Coherence Tomography (OCT) scans, a number of abnormal structural changes in the…
Co-Teaching for Unsupervised Domain Adaptation and Expansion
Hailan Lin, Qijie Wei, Kaibin Tian +2
Unsupervised Domain Adaptation (UDA) essentially trades a model's performance on a source domain for improving its performance on a target domain. To overcome this, Unsupervised Do…
DRAG: Dynamic Region-Aware GCN for Privacy-Leaking Image Detection
Guang Yang, Juan Cao, Qiang Sheng +3
The daily practice of sharing images on social media raises a severe issue about privacy leakage. To address the issue, privacy-leaking image detection is studied recently, with th…
Deepfake Network Architecture Attribution
Tianyun Yang, Ziyao Huang, Juan Cao +2
With the rapid progress of generation technology, it has become necessary to attribute the origin of fake images. Existing works on fake image attribution perform multi-class class…
Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval
Jianfeng Dong, Yabing Wang, Xianke Chen +4
This paper aims for the task of text-to-video retrieval, where given a query in the form of a natural-language sentence, it is asked to retrieve videos which are semantically relev…