10 papers
Revisiting Frame-Wise Saliency for Audio Moment Retrieval
Tatsuya Komatsu, Tatsuya Munakata, Hokuto Munakata
This paper revisits frame-wise saliency for audio moment retrieval (AMR). We show that the frame-wise saliency sequence, conventionally used only as an auxiliary output in DETR-bas…
Overview and Meta-Analysis of DCASE 2026 Challenge Task 6: Audio Moment Retrieval from Long Audio
Hokuto Munakata, Tatsuya Komatsu, Keisuke Imoto +3
This paper presents an overview of the Detection and Classification of Acoustic Scenes and Events (DCASE) 2026 Challenge Task 6, Audio Moment Retrieval (AMR) from Long Audio. Given…
CASTELLA: Long Audio Dataset with Captions and Temporal Boundaries
Hokuto Munakata, Takehiro Imamura, Taichi Nishimura +1
We introduce CASTELLA, a human-annotated audio benchmark for the task of audio moment retrieval (AMR). Although AMR has various useful potential applications, there is still no est…
Hallucination Localization in Video Captioning
Shota Nakada, Kazuhiro Saito, Yuchi Ishikawa +3
We propose a novel task, hallucination localization in video captioning, which aims to identify hallucinations in video captions at the span level (i.e. individual words or phrases…
ProLAP: Probabilistic Language-Audio Pre-Training
Toranosuke Manabe, Yuchi Ishikawa, Hokuto Munakata +1
Language-audio joint representation learning frameworks typically depend on deterministic embeddings, assuming a one-to-one correspondence between audio and text. In real-world set…
Language-Guided Contrastive Audio-Visual Masked Autoencoder with Automatically Generated Audio-Visual-Text Triplets from Videos
Yuchi Ishikawa, Shota Nakada, Hokuto Munakata +3
In this paper, we propose Language-Guided Contrastive Audio-Visual Masked Autoencoders (LG-CAV-MAE) to improve audio-visual representation learning. LG-CAV-MAE integrates a pretrai…
DETECLAP: Enhancing Audio-Visual Representation Learning with Object Information
Shota Nakada, Taichi Nishimura, Hokuto Munakata +2
Current audio-visual representation learning can capture rough object categories (e.g., ``animals'' and ``instruments''), but it lacks the ability to recognize fine-grained details…
Language-based Audio Moment Retrieval
Hokuto Munakata, Taichi Nishimura, Shota Nakada +1
In this paper, we propose and design a new task called audio moment retrieval (AMR). Unlike conventional language-based audio retrieval tasks that search for short audio clips from…
Lighthouse: A User-Friendly Library for Reproducible Video Moment Retrieval and Highlight Detection
Taichi Nishimura, Shota Nakada, Hokuto Munakata +1
We propose Lighthouse, a user-friendly library for reproducible video moment retrieval and highlight detection (MR-HD). Although researchers proposed various MR-HD approaches, the…
Song Data Cleansing for End-to-End Neural Singer Diarization Using Neural Analysis and Synthesis Framework
Hokuto Munakata, Ryo Terashima, Yusuke Fujita
We propose a data cleansing method that utilizes a neural analysis and synthesis (NANSY++) framework to train an end-to-end neural diarization model (EEND) for singer diarization.…