14 papers
The Devil is in the Spectrum Bias: Spectrum-Balanced Feature Matching for Robust Representation Distillation
Kuniaki Saito, Yoshitaka Ushiku
Large visual foundation models have demonstrated remarkable transferability across a wide range of downstream tasks. To deploy such models efficiently, feature matching has become…
VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration
Changbeen Kim, Junwon Chang, Kipyo Kim +3
While Video Large Language Models (Video-LLMs) have recently demonstrated strong performance, reliably evaluating their fine-grained video understanding remains challenging. Existi…
CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding
Yuri Ishitoya, Jeremy Siburian, Masashi Hamaya +3
Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the…
GAVEL: Grounded Caption Error Verification and Localization
Zixian Gao, Atsushi Hashimoto, Kuniaki Saito
Vision-language models (VLMs) often produce hallucinated or inconsistent outputs, where text and images are not properly aligned. Addressing this issue requires not only detecting…
BioVITA: Biological Dataset, Model, and Benchmark for Visual-Textual-Acoustic Alignment
Risa Shinoda, Kaede Shiohara, Nakamasa Inoue +3
Understanding animal species from multimodal data poses an emerging challenge at the intersection of computer vision and ecology. While recent biological models, such as BioCLIP, h…
HalDec-Bench: Benchmarking Hallucination Detector in Image Captioning
Kuniaki Saito, Risa Shinoda, Shohei Tanaka +3
Hallucination detection in captions (HalDec) assesses a vision-language model's ability to correctly align image content with text by identifying errors in captions that misreprese…
Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Models
Hikaru Asano, Tadashi Kozuno, Kuniaki Saito +1
Masked Diffusion Language Models (MDLMs) generate text by iteratively filling masked tokens, requiring two coupled decisions at each step: which positions to unmask (where-to-unmas…
Towards Safer Mobile Agents: Scalable Generation and Evaluation of Diverse Scenarios for VLMs
Takara Taniguchi, Kuniaki Saito, Atsushi Hashimoto
Vision Language Models (VLMs) are increasingly deployed in autonomous vehicles and mobile systems, making it crucial to evaluate their ability to support safer decision-making in c…
HalDec-Bench: Benchmarking Hallucination Detector in Image Captioning
Kuniaki Saito, Risa Shinoda, Shohei Tanaka +3
Hallucination detection in captions (HalDec) assesses a vision-language model's ability to correctly align image content with text by identifying errors in captions that misreprese…
Training-Free Label Space Alignment for Universal Domain Adaptation
Dujin Lee, Sojung An, Jungmyung Wi +2
Universal domain adaptation (UniDA) transfers knowledge from a labeled source domain to an unlabeled target domain, where label spaces may differ and the target domain may contain…
Learning Contrastive Multimodal Fusion with Improved Modality Dropout for Disease Detection and Prediction
Yi Gu, Kuniaki Saito, Jiaxin Ma
As medical diagnoses increasingly leverage multimodal data, machine learning models are expected to effectively fuse heterogeneous information while remaining robust to missing mod…
CaptionSmiths: Flexibly Controlling Language Pattern in Image Captioning
Kuniaki Saito, Donghyun Kim, Kwanyong Park +2
An image captioning model flexibly switching its language pattern, e.g., descriptiveness and length, should be useful since it can be applied to diverse applications. However, desp…
SBS Figures: Pre-training Figure QA from Stage-by-Stage Synthesized Images
Risa Shinoda, Kuniaki Saito, Shohei Tanaka +2
Building a large-scale figure QA dataset requires a considerable amount of work, from gathering and selecting figures to extracting attributes like text, numbers, and colors, and g…
Weak-to-Strong Compositional Learning from Generative Models for Language-based Object Detection
Kwanyong Park, Kuniaki Saito, Donghyun Kim
Vision-language (VL) models often exhibit a limited understanding of complex expressions of visual objects (e.g., attributes, shapes, and their relations), given complex and divers…