12 citations · 23 across the 16 of their papers we have counts for
17 papers
Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection
Atsuyuki Miyai, Kiyoharu Aizawa, Toshihiko Yamasaki
We present a novel approach to efficient LLM harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on va…
360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agents
Kenta Watanabe, Atsuyuki Miyai, Mizuki Takenawa +2
We present 360CityArena, a benchmark for evaluating the urban exploration capabilities of embodied agents within a photorealistic environment constructed from 360-degree videos. Ex…
Manga109-v2026: Revisiting Manga109 Annotations for Modern Manga Understanding
Jeonghun Baek, Atsuyuki Miyai, Shota Onohara +2
Manga is a culturally distinctive multimodal medium and one of the most influential forms of Japanese popular culture. As AI systems increasingly target manga understanding, OCR, a…
Paper Reconstruction Evaluation: Evaluating Presentation and Hallucination in AI-written Papers
Atsuyuki Miyai, Mashiro Toyooka, Zaiying Zhao +3
This paper introduces the first systematic evaluation framework for quantifying the quality and risks of papers written by modern coding agents. While AI-driven paper writing has b…
JMMMU-Pro: Image-based Japanese Multi-discipline Multimodal Understanding Benchmark via Vibe Benchmark Construction
Atsuyuki Miyai, Shota Onohara, Jeonghun Baek +1
This paper introduces JMMMU-Pro, an image-based Japanese Multi-discipline Multimodal Understanding Benchmark, and Vibe Benchmark Construction, a scalable construction method. Follo…
Jr. AI Scientist and Its Risk Report: Autonomous Scientific Exploration from a Baseline Paper
Atsuyuki Miyai, Mashiro Toyooka, Takashi Otonari +2
Understanding the current capabilities and risks of AI Scientist systems (autoresearch) is essential for ensuring trustworthy and sustainable AI-driven scientific progress while pr…
PULSE: Practical Evaluation Scenarios for Large Multimodal Model Unlearning
Tatsuki Kawakami, Kazuki Egashira, Atsuyuki Miyai +2
In recent years, unlearning techniques, which are methods for inducing a model to "forget" previously learned information, have attracted attention as a way to address privacy and…
WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks
Atsuyuki Miyai, Zaiying Zhao, Kazuki Egashira +9
Powered by large language models (LLMs), web browsing agents operate graphical user interfaces in a human-like manner, offering a transparent and general framework for automating w…
MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal Manga Understanding
Jeonghun Baek, Kazuki Egashira, Shota Onohara +4
Manga, or Japanese comics, is a richly multimodal narrative form that blends images and text in complex ways. Teaching large multimodal models (LMMs) to understand such narratives…
A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Models
Shiho Noda, Atsuyuki Miyai, Qing Yu +2
Out-of-distribution (OOD) detection is a task that detects OOD samples during inference to ensure the safety of deployed models. However, conventional benchmarks have reached perfo…
JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation
Shota Onohara, Atsuyuki Miyai, Yuki Imajuku +5
Accelerating research on Large Multimodal Models (LMMs) in non-English languages is crucial for enhancing user experiences across broader populations. In this paper, we introduce J…
Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey
Atsuyuki Miyai, Jingkang Yang, Jingyang Zhang +10
Detecting out-of-distribution (OOD) samples is crucial for ensuring the safety of machine learning systems and has shaped the field of OOD detection. Meanwhile, several other probl…
Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models
Atsuyuki Miyai, Jingkang Yang, Jingyang Zhang +7
This paper introduces a novel task to evaluate the robust understanding capability of Large Multimodal Models (LMMs), termed . Multiple…
Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?
Atsuyuki Miyai, Qing Yu, Go Irie +1
Out-of-distribution (OOD) detection is critical for safety-sensitive machine learning applications and has been extensively studied, yielding a plethora of methods developed in the…
LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning
Atsuyuki Miyai, Qing Yu, Go Irie +1
We present a novel vision-language prompt learning approach for few-shot out-of-distribution (OOD) detection. Few-shot OOD detection aims to detect OOD images from classes that are…
GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detection
Atsuyuki Miyai, Qing Yu, Go Irie +1
Zero-shot out-of-distribution (OOD) detection is a task that detects OOD images during inference with only in-distribution (ID) class names. Existing methods assume ID images conta…
Rethinking Rotation in Self-Supervised Contrastive Learning: Adaptive Positive or Negative Data Augmentation
Atsuyuki Miyai, Qing Yu, Daiki Ikami +2
Rotation is frequently listed as a candidate for data augmentation in contrastive learning but seldom provides satisfactory improvements. We argue that this is because the rotated…