7 citations · 25 across the 13 of their papers we have counts for
15 papers
Linear Fusion MultiDiffusion for Fast Training-Free Spherical Panorama Generation
Akio Hayakawa, Yusuke Mukuta, Tatsuya Harada
We propose LF-MultiDiffusion, a training-free panorama generation method that extends MultiDiffusion to support linear projections between target and reference image spaces. Our ke…
Echoes Over Time: Unlocking Length Generalization in Video-to-Audio Generation Models
Christian Simon, Masato Ishii, Wei-Yao Wang +8
Scaling multimodal alignment between video and audio is challenging, particularly due to limited data and the mismatch between text descriptions and frame-level video information.…
AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Path
Zhengyang Yu, Akio Hayakawa, Masato Ishii +4
Autoregressive video diffusion models (AR-VDMs) show strong promise as scalable alternatives to bidirectional VDMs, enabling real-time and interactive applications. Yet there remai…
Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Edits
Masato Ishii, Akio Hayakawa, Takashi Shibuya +1
We introduce a novel pipeline for joint audio-visual editing that enhances the coherence between edited video and its accompanying audio. Our approach first applies state-of-the-ar…
SoundReactor: Frame-level Online Video-to-Audio Generation
Koichi Saito, Julian Tanke, Christian Simon +7
Prevailing Video-to-Audio (V2A) generation models operate offline, assuming an entire video sequence or chunks of frames are available beforehand. This critically limits their use…
TITAN-Guide: Taming Inference-Time AligNment for Guided Text-to-Video Diffusion Models
Christian Simon, Masato Ishii, Akio Hayakawa +4
In the recent development of conditional diffusion models still require heavy supervised fine-tuning for performing control on a category of tasks. Training-free conditioning via g…
Step-by-Step Video-to-Audio Synthesis via Negative Audio Guidance
Akio Hayakawa, Masato Ishii, Takashi Shibuya +1
We propose a step-by-step video-to-audio (V2A) generation method that provides finer control over the generation process and more realistic audio synthesis. Inspired by traditional…
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis
Ho Kei Cheng, Masato Ishii, Akio Hayakawa +3
We propose to synthesize high-quality and synchronized audio, given video and optional text conditions, using a novel multimodal joint training framework MMAudio. In contrast to si…
A Simple but Strong Baseline for Sounding Video Generation: Effective Adaptation of Audio and Video Diffusion Models for Joint Generation
Masato Ishii, Akio Hayakawa, Takashi Shibuya +1
In this work, we build a simple but strong baseline for sounding video generation. Given base diffusion models for audio and video, we integrate them with additional modules into a…
MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation
Akio Hayakawa, Masato Ishii, Takashi Shibuya +1
This study aims to construct an audio-video generative model with minimal computational cost by leveraging pre-trained single-modal generative models for audio and video. To achiev…
Instruct 3D-to-3D: Text Instruction Guided 3D-to-3D conversion
Hiromichi Kamata, Yuiko Sakuma, Akio Hayakawa +2
We propose a high-quality 3D-to-3D conversion method, Instruct 3D-to-3D. Our method is designed for a novel task, which is to convert a given 3D scene to another scene according to…
Fine-grained Image Editing by Pixel-wise Guidance Using Diffusion Models
Naoki Matsunaga, Masato Ishii, Akio Hayakawa +2
Our goal is to develop fine-grained real-image editing methods suitable for real-world applications. In this paper, we first summarize four requirements for these methods and propo…
Neural Network Libraries: A Deep Learning Framework Designed from Engineers' Perspectives
Takuya Narihira, Javier Alonsogarcia, Fabien Cardinaux +14
While there exist a plethora of deep learning tools and frameworks, the fast-growing complexity of the field brings new demands and challenges, such as more flexible network design…
Reference-Based Video Colorization with Spatiotemporal Correspondence
Naofumi Akimoto, Akio Hayakawa, Andrew Shin +1
We propose a novel reference-based video colorization framework with spatiotemporal correspondence. Reference-based methods colorize grayscale frames referencing a user input color…
Out-of-core Training for Extremely Large-Scale Neural Networks With Adaptive Window-Based Scheduling
Akio Hayakawa, Takuya Narihira
While large neural networks demonstrate higher performance in various tasks, training large networks is difficult due to limitations on GPU memory size. We propose a novel out-of-c…