10 citations · 35 across the 26 of their papers we have counts for
31 papers
Representing short-term movement to reveal animal identity and learning progress
Ping Zhang, Quang-Huy Nguyen, Yingtie Lei +5
How learning progresses over time and varies across individuals is a longstanding question in cognitive and behavioral science. However, a major challenge is identifying observable…
Ontic state recovery through prediction
Ping Zhang, Quang-Huy Nguyen, Yingtie Lei +1
We encounter a system through its responses, yet its state concerns how the system is. Predicting further responses promises knowledge beyond what is immediately observed. What tha…
Mind the Tail: Data Selection Through Sparse Concepts
Ziheng Zhang, Neha Hulkund, Colin Lee +7
Data selection reallocates training support across visual concepts, including objects, attributes, color, texture, and backgrounds. Different allocation strategies may produce vari…
Probing Identity-Specific Motion Signatures: A Controlled Diagnostic Study
Yingtie Lei, Fangxun Liu, Baicheng Wu +13
Identity recognition (e.g., person, animal re-identification) has traditionally relied heavily on static appearance cues. Yet motion--consistent, individual-specific dynamics--can…
Finer-Personalization Rank: Fine-Grained Retrieval Examines Identity Preservation for Personalized Generation
Connor Kilrain, David Carlyn, Julia Chae +3
The rise of personalized generative models raises a central question: how should we evaluate identity preservation? Given a reference image (e.g., one's pet), we expect the generat…
BeetleFlow: An Integrative Deep Learning Pipeline for Beetle Image Processing
Fangxun Liu, S M Rayeed, Samuel Stevens +21
In entomology and ecology research, biologists often need to collect a large number of insects, among which beetles are the most common species. A common practice for biologists to…
BioCAP: Exploiting Synthetic Captions Beyond Labels in Biological Foundation Models
Ziheng Zhang, Xinyue Ma, Arpita Chowdhury +9
This work investigates descriptive captions as an additional source of supervision for biological multimodal foundation models. Images and captions can be viewed as complementary s…
A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime
Shuning Jiang, Wei-Lun Chao, Daniel Haehn +2
We present a data-domain sampling regime for quantifying CNNs' graphic perception behaviors. This regime lets us evaluate CNNs' ratio estimation ability in bar charts from three pe…
AVA-Bench: Atomic Visual Ability Benchmark for Vision Foundation Models
Zheda Mai, Arpita Chowdhury, Zihe Wang +5
The rise of vision foundation models (VFMs) calls for systematic evaluation. A common approach pairs VFMs with large language models (LLMs) as general-purpose heads, followed by ev…
BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning
Jianyang Gu, Samuel Stevens, Elizabeth G Campolongo +13
Foundation models trained at scale exhibit remarkable emergent behaviors, learning new capabilities beyond their initial training objectives. We find such emergent behaviors in bio…
Building Machine Learning Challenges for Anomaly Detection in Science
Elizabeth G. Campolongo, Yuan-Tang Chou, Ekaterina Govorkova +148
Scientific discoveries are often made by finding a pattern or object that was not predicted by the known rules of science. Oftentimes, these anomalous events or objects that do not…
Federated Inverse Probability Treatment Weighting for Individual Treatment Effect Estimation
Changchang Yin, Hong-You Chen, Wei-Lun Chao +1
Individual treatment effect (ITE) estimation is to evaluate the causal effects of treatment strategies on some important outcomes, which is a crucial problem in healthcare. Most ex…
Revisiting semi-supervised learning in the era of foundation models
Ping Zhang, Zheda Mai, Quang-Huy Nguyen +1
Semi-supervised learning (SSL) leverages abundant unlabeled data alongside limited labeled data to enhance learning. As vision foundation models (VFMs) increasingly serve as the ba…
Interpretable and Testable Vision Features via Sparse Autoencoders
Samuel Stevens, Wei-Lun Chao, Tanya Berger-Wolf +1
To truly understand vision models, we must not only interpret their learned features but also validate these interpretations through controlled experiments. While earlier work offe…
Finer-CAM: Spotting the Difference Reveals Finer Details for Visual Explanation
Ziheng Zhang, Jianyang Gu, Arpita Chowdhury +5
Class activation map (CAM) has been widely used to highlight image regions that contribute to class predictions. Despite its simplicity and computational efficiency, CAM often stru…
Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysis
Arpita Chowdhury, Dipanjyoti Paul, Zheda Mai +10
We present a simple approach to make pre-trained Vision Transformers (ViTs) interpretable for fine-grained analysis, aiming to identify and localize the traits that distinguish vis…
Static Segmentation by Tracking: A Label-Efficient Approach for Fine-Grained Specimen Image Segmentation
Zhenyang Feng, Zihe Wang, Jianyang Gu +22
We study image segmentation in the biological domain, particularly trait segmentation from specimen images (e.g., butterfly wing stripes, beetle elytra). This fine-grained task is…
Fine-Tuning is Fine, if Calibrated
Zheda Mai, Arpita Chowdhury, Ping Zhang +8
Fine-tuning is arguably the most straightforward way to tailor a pre-trained model (e.g., a foundation model) to downstream applications, but it also comes with the risk of losing…
Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition
Zheda Mai, Ping Zhang, Cheng-Hao Tu +3
Parameter-efficient fine-tuning (PEFT) has attracted significant attention due to the growth of pre-trained model sizes and the need to fine-tune (FT) them for superior downstream…
What Do You See in Common? Learning Hierarchical Prototypes over Tree-of-Life to Discover Evolutionary Traits
Harish Babu Manogaran, M. Maruf, Arka Daw +12
A grand challenge in biology is to discover evolutionary traits - features of organisms common to a group of species with a shared ancestor in the tree of life (also referred to as…
VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images
M. Maruf, Arka Daw, Kazi Sajeed Mehrab +19
Images are increasingly becoming the currency for documenting biodiversity on the planet, providing novel opportunities for accelerating scientific discoveries in the field of orga…
Hierarchical Conditioning of Diffusion Models Using Tree-of-Life for Studying Species Evolution
Mridul Khurana, Arka Daw, M. Maruf +12
A central problem in biology is to understand how organisms evolve and adapt to their environment by acquiring variations in the observable characteristics or traits of species acr…
Jigsaw Game: Federated Clustering
Jinxuan Xu, Hong-You Chen, Wei-Lun Chao +1
Federated learning has recently garnered significant attention, especially within the domain of supervised learning. However, despite the abundance of unlabeled data on end-users,…
MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs
Jihyung Kil, Zheda Mai, Justin Lee +6
The ability to compare objects, scenes, or situations is crucial for effective decision-making and problem-solving in everyday life. For instance, comparing the freshness of apples…