77 citations · 167 across the 34 of their papers we have counts for
36 papers
MEDit-Bench: A Dataset for Evaluating Message-Driven Narrative Video Editing
Katsuya Ogata, Zongshang Pang, Mayu Otani +1
Video editing is fundamentally message-driven: even from the same source footage, the selected shots change depending on the narrative the editor wishes to convey. Benchmarks for a…
DiverXplorer: Stock Image Exploration via Diversity Adjustment for Graphic Design
Antonio Tejero-de-Pablos, Sichao Song, Naoto Ohsaka +2
Graphic designers explore large stock image collections during open-ended or early-stage design tasks, yet common tools emphasize relevance and similarity, limiting designers' abil…
Evaluating Cross-Modal Reasoning Ability and Problem Characteristics with Multimodal Item Response Theory
Shunki Uebayashi, Kento Masui, Kyohei Atarashi +5
Multimodal Large Language Models (MLLMs) have recently emerged as general architectures capable of reasoning over diverse modalities. Benchmarks for MLLMs should measure their abil…
Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMs
Zongshang Pang, Mayu Otani, Yuta Nakashima
Temporally localizing user-queried events through natural language is a crucial capability for video models. Recent methods predominantly adapt video LLMs to generate event boundar…
Harnessing the Latent Diffusion Model for Training-Free Image Style Transfer
Kento Masui, Mayu Otani, Masahiro Nomura +1
Diffusion models have recently shown the ability to generate high-quality images. However, controlling its generation process still poses challenges. The image style transfer task…
Multimodal Markup Document Models for Graphic Design Completion
Kotaro Kikuchi, Ukyo Honda, Naoto Inoue +3
We introduce MarkupDM, a multimodal markup document model that represents graphic design as an interleaved multimodal document consisting of both markup language and images. Unlike…
LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generation
Mayu Otani, Naoto Inoue, Kotaro Kikuchi +1
We introduce a layout similarity measure designed to evaluate the results of layout generation. While several similarity measures have been proposed in prior research, there has be…
Would Deep Generative Models Amplify Bias in Future Models?
Tianwei Chen, Yusuke Hirota, Mayu Otani +2
We investigate the impact of deep generative models on potential social biases in upcoming computer vision models. As the internet witnesses an increasing influx of AI-generated im…
LayoutFlow: Flow Matching for Layout Generation
Julian Jorge Andrade Guerreiro, Naoto Inoue, Kento Masui +2
Finding a suitable layout represents a crucial task for diverse applications in graphic design. Motivated by simpler and smoother sampling trajectories, we explore the use of Flow…
Towards Artwork Explanation in Large-scale Vision Language Models
Kazuki Hayashi, Yusuke Sakai, Hidetaka Kamigaito +2
Large-scale Vision-Language Models (LVLMs) output text from images and instructions, demonstrating capabilities in text generation and comprehension. However, it has not been clari…
Multimodal Color Recommendation in Vector Graphic Documents
Qianru Qiu, Xueting Wang, Mayu Otani
Color selection plays a critical role in graphic document design and requires sufficient consideration of various contexts. However, recommending appropriate colors which harmonize…
Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation
Mayu Otani, Riku Togashi, Yu Sawai +5
Human evaluation is critical for validating the performance of text-to-image generative models, as this highly cognitive process requires deep comprehension of text and images. How…
Towards Flexible Multi-modal Document Models
Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra +2
Creative workflows for generating graphical documents involve complex inter-related tasks, such as aligning elements, choosing appropriate fonts, or employing aesthetically harmoni…
LayoutDM: Discrete Diffusion Model for Controllable Layout Generation
Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra +2
Controllable layout generation aims at synthesizing plausible arrangement of element bounding boxes with optional constraints, such as type or position of a specific element. In th…
Generative Colorization of Structured Mobile Web Pages
Kotaro Kikuchi, Naoto Inoue, Mayu Otani +2
Color is a critical design factor for web pages, affecting important factors such as viewer emotions and the overall trust and satisfaction of a website. Effective coloring require…
Contrastive Losses Are Natural Criteria for Unsupervised Video Summarization
Zongshang Pang, Yuta Nakashima, Mayu Otani +1
Video summarization aims to select the most informative subset of frames in a video to facilitate efficient video browsing. Unsupervised methods usually rely on heuristic training…
Video Summarization Overview
Mayu Otani, Yale Song, Yang Wang
With the broad growth of video capturing devices and applications on the web, it is more demanding to provide desired video content for users efficiently. Video summarization facil…
Color Recommendation for Vector Graphic Documents based on Multi-Palette Representation
Qianru Qiu, Xueting Wang, Mayu Otani +1
Vector graphic documents present multiple visual elements, such as images, shapes, and texts. Choosing appropriate colors for multiple visual elements is a difficult but crucial ta…
Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks
Tianwei Chen, Noa Garcia, Mayu Otani +3
Is more data always better to train vision-and-language models? We study knowledge transferability in multi-modal tasks. The current tendency in machine learning is to assume that…
Does Robustness on ImageNet Transfer to Downstream Tasks?
Yutaro Yamada, Mayu Otani
As clean ImageNet accuracy nears its ceiling, the research community is increasingly more concerned about robust accuracy under distributional shifts. While a variety of methods ha…
AxIoU: An Axiomatically Justified Measure for Video Moment Retrieval
Riku Togashi, Mayu Otani, Yuta Nakashima +3
Evaluation measures have a crucial impact on the direction of research. Therefore, it is of utmost importance to develop appropriate and reliable evaluation measures for new applic…
Optimal Correction Cost for Object Detection Evaluation
Mayu Otani, Riku Togashi, Yuta Nakashima +3
Mean Average Precision (mAP) is the primary evaluation measure for object detection. Although object detection has a broad range of applications, mAP evaluates detectors in terms o…
Transferring Domain-Agnostic Knowledge in Video Question Answering
Tianran Wu, Noa Garcia, Mayu Otani +3
Video question answering (VideoQA) is designed to answer a given question based on a relevant video clip. The current available large-scale datasets have made it possible to formul…
Constrained Graphic Layout Generation via Latent Optimization
Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani +1
It is common in graphic design humans visually arrange various elements according to their design intent and semantics. For example, a title text almost always appears on top of ot…