Synthetic Boost: Leveraging Synthetic Data for Enhanced Vision-Language Segmentation in Echocardiography

Abstract

Accurate segmentation is essential for echocardiography-based assessment of cardiovascular diseases (CVDs). However, the variability among sonographers and the inherent challenges of ultrasound images hinder precise segmentation. By leveraging the joint representation of image and text modalities, Vision-Language Segmentation Models (VLSMs) can incorporate rich contextual information, potentially aiding in accurate and explainable segmentation. However, the lack of readily available data in echocardiography hampers the training of VLSMs. In this study, we explore using synthetic datasets from Semantic Diffusion Models (SDMs) to enhance VLSMs for echocardiography segmentation. We evaluate results for two popular VLSMs (CLIPSeg and CRIS) using seven different kinds of language prompts derived from several attributes, automatically extracted from echocardiography images, segmentation masks, and their metadata. Our results show improved metrics and faster convergence when pretraining VLSMs on SDM-generated synthetic images before finetuning on real images. The code, configs, and prompts are available at https://github.com/naamiinepal/synthetic-boost.

arXiv commentAccepted at the 4th International Workshop of Advances in Simplifying Medical UltraSound (ASMUS)

References in corpus (7)

• Automated Design of Deep Learning Methods for Biomedical Image Segmentation
• Learning Transferable Visual Models From Natural Language Supervision
• Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
• Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography
• OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
• Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm
• CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP