765 citations · 1.9k across the 87 of their papers we have counts for
9 papers · 1 filter
Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs
Yiwei Li, Yifan Zhou, Huaqin Zhao +6
Vision--language models (VLMs) process images as visual tokens, yet their intermediate reasoning is often carried out in text, which can be suboptimal for visually grounded radiolo…
OracleSage: Towards Unified Visual-Linguistic Understanding of Oracle Bone Scripts through Cross-Modal Knowledge Fusion
Hanqi Jiang, Yi Pan, Junhao Chen +8
Oracle bone script (OBS), as China's earliest mature writing system, present significant challenges in automatic recognition due to their complex pictographic structures and diverg…
Examining the Commitments and Difficulties Inherent in Multimodal Foundation Models for Street View Imagery
Zhenyuan Yang, Xuhui Lin, Qinyi He +10
The emergence of Large Language Models (LLMs) and multimodal foundation models (FMs) has generated heightened interest in their applications that integrate vision and language. Thi…
Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning
Chong Ma, Hanqi Jiang, Wenting Chen +10
In the medical multi-modal frameworks, the alignment of cross-modality features presents a significant challenge. However, existing works have learned features that are implicitly…
On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications
Chenjiao Tan, Qian Cao, Yiwei Li +15
The advent of large language models (LLMs) has heightened interest in their potential for multimodal applications that integrate language and vision. This paper explores the capabi…
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation
Cheng Chen, Juzheng Miao, Dufan Wu +10
The Segment Anything Model (SAM), a foundation model for general image segmentation, has demonstrated impressive zero-shot performance across numerous natural image segmentation ta…
Hierarchical Semantic Tree Concept Whitening for Interpretable Image Classification
Haixing Dai, Lu Zhang, Lin Zhao +9
With the popularity of deep neural networks (DNNs), model interpretability is becoming a critical concern. Many approaches have been developed to tackle the problem through post-ho…
Review of Large Vision Models and Visual Prompt Engineering
Jiaqi Wang, Zhengliang Liu, Lin Zhao +18
Visual prompt engineering is a fundamental technology in the field of visual and image Artificial General Intelligence, serving as a key component for achieving zero-shot capabilit…
SAMAug: Point Prompt Augmentation for Segment Anything Model
Haixing Dai, Chong Ma, Zhiling Yan +14
This paper introduces SAMAug, a novel visual point augmentation method for the Segment Anything Model (SAM) that enhances interactive image segmentation performance. SAMAug generat…