5 papers
Beyond Flat Labels: Level-Restricted Contrastive Learning for Hierarchical Fine-Grained Vision Classification
Zhiyuan Tao, Srikumar Sastry, Matthew J Thompson +9
Multimodal contrastive learning has enabled zero-shot visual classification by aligning images with textual categories. However, in hierarchically structured label spaces, existing…
Towards Open-Ended Visual Scientific Discovery with Sparse Autoencoders
Samuel Stevens, Jacob Beattie, Tanya Berger-Wolf +2
Foundation models in several scientific domains, including visual domains, learn representations that capture complex semantics from their respective fields. Despite this, most exi…
BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models
Ziheng Zhang, Xinyue Ma, Arpita Chowdhury +9
This work investigates descriptive captions as an additional source of supervision for biological multimodal foundation models. Images and captions can be viewed as complementary s…
Measuring Information Richness in Product Images: Implications for Online Sales
Zhu Yuting, Cao Xinyu, Su Yuzhuo +1
A common challenge for e-commerce sellers is to decide what product images to display on online shopping sites. In this paper, we propose and validate a novel metric, k-value, to q…
Static Segmentation by Tracking: A Label-Efficient Approach for Fine-Grained Specimen Image Segmentation
Zhenyang Feng, Zihe Wang, Jianyang Gu +22
We study image segmentation in the biological domain, particularly trait segmentation from specimen images (e.g., butterfly wing stripes, beetle elytra). This fine-grained task is…