13 citations · 41 across the 41 of their papers we have counts for
49 papers
Disen-Forcing: Disentangling Semantic Anchoring from Motion for Autoregressive Video Diffusion
Zhengyang Yu, Akio Hayakawa, Masato Ishii +5
Autoregressive video diffusion models (AR-VDMs) enable real-time, long-horizon video generation but degrade rapidly once the inference horizon outruns the training horizon, due to…
AVIO: Learning to Add and Remove Sounding Objects in Audiovisual Scenes
Weihan Xu, Kan Jen Cheng, Koichi Saito +8
Adding or removing a sounding object requires coordinated changes to visual content and sound while preserving the surrounding scene. Yet paired supervision for localized non-speec…
LYRIC: Language-Driven Physics-Based Character Control for Contact-Rich Whole-Body Object Interaction
Zeyu Han, Zichong Meng, Julian Tanke +8
We present LYRIC, a generative flow-matching controller for language-driven physics-based contact-rich interaction control, that enables simulated characters to perform contact-ric…
Omni-Interactive Universal Embedder
Wei-Yao Wang, Kazuya Tateishi, Shuyang Cui +4
Multimodal representation learning has been shifting from traditional two-tower architectures to large language model (LLM)-based embedders due to their strong instruction-followin…
Spectral Prior for Reducing Exposure Bias in Diffusion Models
Yuya Kobayashi, Masato Ishii, Yuhta Takida +2
Diffusion models typically suffer from error accumulation during iterative sampling, commonly referred to as exposure bias. We reveal systematic frequency-dependent discrepancies b…
Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion
Dongseok Shim, Julian Tanke, Kengo Uchida +5
Human motion generation has been widely studied across diverse input modalities, text, music, and video, and recent efforts have unified these into single multimodal frameworks. Ho…
Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources
Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida +6
Sound events have semantic identities, locations, and trajectories, yet existing audio-language models typically encode clips as global event content. Conversely, sound event local…
Echoes Over Time: Unlocking Length Generalization in Video-to-Audio Generation Models
Christian Simon, Masato Ishii, Wei-Yao Wang +8
Scaling multimodal alignment between video and audio is challenging, particularly due to limited data and the mismatch between text descriptions and frame-level video information.…
AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Path
Zhengyang Yu, Akio Hayakawa, Masato Ishii +4
Autoregressive video diffusion models (AR-VDMs) show strong promise as scalable alternatives to bidirectional VDMs, enabling real-time and interactive applications. Yet there remai…
Schrodinger Audio-Visual Editor: Object-Level Audiovisual Removal
Weihan Xu, Kan Jen Cheng, Koichi Saito +10
Joint editing of audio and visual content is crucial for precise and controllable content creation. This new task poses challenges due to the limitations of paired audio-visual dat…
Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Edits
Masato Ishii, Akio Hayakawa, Takashi Shibuya +1
We introduce a novel pipeline for joint audio-visual editing that enhances the coherence between edited video and its accompanying audio. Our approach first applies state-of-the-ar…
Towards Blind Data Cleaning: A Case Study in Music Source Separation
Azalea Gui, Woosung Choi, Junghyun Koo +5
The performance of deep learning models for music source separation heavily depends on training data quality. However, datasets are often corrupted by difficult-to-detect artifacts…
StereoSync: Spatially-Aware Stereo Audio Generation from Video
Christian Marinoni, Riccardo Fosco Gramaccioni, Kazuki Shimada +3
Although audio generation has been widely studied over recent years, video-aligned audio generation still remains a relatively unexplored frontier. To address this gap, we introduc…
SONA: Learning Conditional, Unconditional, and Mismatching-Aware Discriminator
Yuhta Takida, Satoshi Hayakawa, Takashi Shibuya +6
Deep generative models have made significant advances in generating complex content, yet conditional generation remains a fundamental challenge. Existing conditional generative adv…
SoundReactor: Frame-level Online Video-to-Audio Generation
Koichi Saito, Julian Tanke, Christian Simon +7
Prevailing Video-to-Audio (V2A) generation models operate offline, assuming an entire video sequence or chunks of frames are available beforehand. This critically limits their use…
TITAN-Guide: Taming Inference-Time AligNment for Guided Text-to-Video Diffusion Models
Christian Simon, Masato Ishii, Akio Hayakawa +4
In the recent development of conditional diffusion models still require heavy supervised fine-tuning for performing control on a category of tasks. Training-free conditioning via g…
Stereo Sound Event Localization and Detection with Onscreen/offscreen Classification
Kazuki Shimada, Archontis Politis, Iran R. Roman +10
This paper presents the objective, dataset, baseline, and metrics of Task 3 of the DCASE2025 Challenge on sound event localization and detection (SELD). In previous editions, the c…
Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry
Junyoung Seo, Jisang Han, Jaewoo Jung +9
We introduce Vid-CamEdit, a novel framework for video camera trajectory editing, enabling the re-synthesis of monocular videos along user-defined camera paths. This task is challen…
Efficiency without Compromise: CLIP-aided Text-to-Image GANs with Increased Diversity
Yuya Kobayashi, Yuhta Takida, Takashi Shibuya +1
Recently, Generative Adversarial Networks (GANs) have been successfully scaled to billion-scale large text-to-image datasets. However, training such models entails a high training…
Step-by-Step Video-to-Audio Synthesis via Negative Audio Guidance
Akio Hayakawa, Masato Ishii, Takashi Shibuya +1
We propose a step-by-step video-to-audio (V2A) generation method that provides finer control over the generation process and more realistic audio synthesis. Inspired by traditional…
Dyadic Mamba: Long-term Dyadic Human Motion Synthesis
Julian Tanke, Takashi Shibuya, Kengo Uchida +2
Generating realistic dyadic human motion from text descriptions presents significant challenges, particularly for extended interactions that exceed typical training sequence length…
Forging and Removing Latent-Noise Diffusion Watermarks Using a Single Image
Anubhav Jain, Yuya Kobayashi, Naoki Murata +6
Watermarking techniques are vital for protecting intellectual property and preventing fraudulent use of media. Most previous watermarking schemes designed for diffusion models embe…
HumanGif: Single-View Human Diffusion with Generative Prior
Shoukang Hu, Takuya Narihira, Kazumi Fukuda +3
Previous 3D human creation methods have made significant progress in synthesizing view-consistent and temporally aligned results from sparse-view images or monocular videos. Howeve…
CCStereo: Audio-Visual Contextual and Contrastive Learning for Binaural Audio Generation
Yuanhong Chen, Kazuki Shimada, Christian Simon +3
Binaural audio generation (BAG) aims to convert monaural audio to stereo audio using visual prompts, requiring a deep understanding of spatial and semantic information. However, cu…