195 citations · 495 across the 35 of their papers we have counts for
10 papers · 1 filter
DENEB: A Hallucination-Robust Automatic Evaluation Metric for Image Captioning
Kazuki Matsuda, Yuiga Wada, Komei Sugiura
In this work, we address the challenge of developing automatic evaluation metrics for image captioning, with a particular focus on robustness against hallucinations. Existing metri…
Layer-Wise Relevance Propagation with Conservation Property for ResNet
Seitaro Otsuki, Tsumugi Iida, Félix Doublet +4
The transparent formulation of explanation methods is essential for elucidating the predictions of neural networks, which are typically black-box models. Layer-wise Relevance Propa…
Polos: Multimodal Metric Learning from Human Feedback for Image Captioning
Yuiga Wada, Kanta Kaneda, Daichi Saito +1
Establishing an automatic evaluation metric that closely aligns with human judgments is essential for effectively developing image captioning models. Recent data-driven metrics hav…
DialMAT: Dialogue-Enabled Transformer with Moment-Based Adversarial Training
Kanta Kaneda, Ryosuke Korekata, Yuiga Wada +7
This paper focuses on the DialFRED task, which is the task of embodied instruction following in a setting where an agent can actively ask questions about the task. To address this…
JaSPICE: Automatic Evaluation Metric Using Predicate-Argument Structures for Image Captioning Models
Yuiga Wada, Kanta Kaneda, Komei Sugiura
Image captioning studies heavily rely on automatic evaluation metrics such as BLEU and METEOR. However, such n-gram-based metrics have been shown to correlate poorly with human eva…
Multimodal Diffusion Segmentation Model for Object Segmentation from Manipulation Instructions
Yui Iioka, Yu Yoshida, Yuiga Wada +2
In this study, we aim to develop a model that comprehends a natural language instruction (e.g., "Go to the living room and get the nearest pillow to the radio art on the wall") and…
LatteGAN: Visually Guided Language Attention for Multi-Turn Text-Conditioned Image Manipulation
Shoya Matsumori, Yuki Abe, Kosuke Shingyouchi +2
Text-guided image manipulation tasks have recently gained attention in the vision-and-language community. While most of the prior studies focused on single-turn manipulation, our g…
Unified Questioner Transformer for Descriptive Question Generation in Goal-Oriented Visual Dialogue
Shoya Matsumori, Kosuke Shingyouchi, Yuki Abe +3
Building an interactive artificial intelligence that can ask questions about the real world is one of the biggest challenges for vision and language problems. In particular, goal-o…
Alleviating the Burden of Labeling: Sentence Generation by Attention Branch Encoder-Decoder Network
Tadashi Ogura, Aly Magassouba, Komei Sugiura +4
Domestic service robots (DSRs) are a promising solution to the shortage of home care workers. However, one of the main limitations of DSRs is their inability to interact naturally…
Multimodal Attention Branch Network for Perspective-Free Sentence Generation
Aly Magassouba, Komei Sugiura, Hisashi Kawai
In this paper, we address the automatic sentence generation of fetching instructions for domestic service robots. Typical fetching commands such as "bring me the yellow toy from th…