16 citations · 57 across the 24 of their papers we have counts for
24 papers
Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources
Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida +6
Sound events are entities with semantic identities, locations, and trajectories, but current audio-language models usually reason about clips as global event content. Conversely, s…
A Unified View of Score-Based and Drifting Models
Chieh-Hsin Lai, Bac Nguyen, Naoki Murata +5
Drifting models train one-step generators by optimizing a kernel-induced mean-shift discrepancy between the data and model distributions, with Laplace kernels used by default in pr…
Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignment
Bac Nguyen, Yuhta Takida, Naoki Murata +4
Slot Attention (SA) with pretrained diffusion models has recently shown promise for object-centric learning (OCL), but suffers from slot entanglement and weak alignment between obj…
GUDA: Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning
Naoki Murata, Yuhta Takida, Chieh-Hsin Lai +4
Training-data attribution for vision generative models aims to identify which training data influenced a given output. While most methods score individual examples, practitioners o…
PAVAS: Physics-Aware Video-to-Audio Synthesis
Oh Hyun-Bin, Yuhta Takida, Toshimitsu Uesaka +2
Recent advances in Video-to-Audio (V2A) generation have achieved impressive perceptual quality and temporal synchronization, yet most models remain appearance-driven, capturing vis…
Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity
Naoki Yoshida, Satoshi Hayakawa, Yuhta Takida +3
In this study, we propose an enhancement to the similarity computation mechanism in multimodal contrastive pretraining frameworks such as CLIP. Prior theoretical research has demon…
SONA: Learning Conditional, Unconditional, and Mismatching-Aware Discriminator
Yuhta Takida, Satoshi Hayakawa, Takashi Shibuya +6
Deep generative models have made significant advances in generating complex content, yet conditional generation remains a fundamental challenge. Existing conditional generative adv…
Improving Vector-Quantized Image Modeling with Latent Consistency-Matching Diffusion
Bac Nguyen, Chieh-Hsin Lai, Yuhta Takida +4
By embedding discrete representations into a continuous latent space, we can leverage continuous-space latent diffusion models to handle generative modeling of discrete data. Howev…
G2D2: Gradient-Guided Discrete Diffusion for Inverse Problem Solving
Naoki Murata, Chieh-Hsin Lai, Yuhta Takida +4
Recent literature has effectively leveraged diffusion models trained on continuous variables as priors for solving inverse problems. Notably, discrete diffusion models with discret…
DisMix: Disentangling Mixtures of Musical Instruments for Source-level Pitch and Timbre Manipulation
Yin-Jyun Luo, Kin Wai Cheuk, Woosung Choi +8
Existing work on pitch and timbre disentanglement has been mostly focused on single-instrument music audio, excluding the cases where multiple instruments are presented. To fill th…
PaGoDA: Progressive Growing of a One-Step Generator from a Low-Resolution Diffusion Teacher
Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao +5
The diffusion model performs remarkable in generating high-dimensional content but is computationally intensive, especially during training. We propose Progressive Growing of Diffu…
Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metric
Toshimitsu Uesaka, Taiji Suzuki, Yuhta Takida +3
In typical multimodal contrastive learning, such as CLIP, encoders produce one point in the latent representation space for each input. However, one-point representation has diffic…
HQ-VAE: Hierarchical Discrete Representation Learning with Variational Bayes
Yuhta Takida, Yukara Ikemiya, Takashi Shibuya +8
Vector quantization (VQ) is a technique to deterministically learn features with discrete codebook representations. It is commonly performed with a variational autoencoding model,…
Manifold Preserving Guided Diffusion
Yutong He, Naoki Murata, Chieh-Hsin Lai +8
Despite the recent advancements, conditional image generation still faces challenges of cost, generalizability, and the need for task-specific training. In this paper, we propose M…
Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion
Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao +6
Consistency Models (CM) (Song et al., 2023) accelerate score-based diffusion model sampling at the cost of sample quality but lack a natural way to trade-off quality for speed. To…
On the Equivalence of Consistency-Type Models: Consistency Models, Consistent Diffusion Models, and Fokker-Planck Regularization
Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka +3
The emergence of various notions of ``consistency'' in diffusion models has garnered considerable attention and helped achieve improved sample quality, likelihood estimation, and a…
SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer
Yuhta Takida, Masaaki Imaizumi, Takashi Shibuya +4
Generative adversarial networks (GANs) learn a target probability distribution by optimizing a generator and a discriminator with minimax objectives. This paper addresses the quest…
GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restoration
Naoki Murata, Koichi Saito, Chieh-Hsin Lai +4
Pre-trained diffusion models have been successfully used as priors in a variety of linear inverse problems, where the goal is to reconstruct a signal from noisy linear measurements…
Unsupervised vocal dereverberation with diffusion-based generative models
Koichi Saito, Naoki Murata, Toshimitsu Uesaka +4
Removing reverb from reverberant music is a necessary technique to clean up audio for downstream music manipulations. Reverberation of music contains two categories, natural reverb…
DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability
Kin Wai Cheuk, Ryosuke Sawata, Toshimitsu Uesaka +5
In this paper we propose a novel generative approach, DiffRoll, to tackle automatic music transcription (AMT). Instead of treating AMT as a discriminative task in which the model i…
Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement
Ryosuke Sawata, Naoki Murata, Yuhta Takida +4
Although deep neural network (DNN)-based speech enhancement (SE) methods outperform the previous non-DNN-based ones, they often degrade the perceptual quality of generated outputs.…
FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation
Chieh-Hsin Lai, Yuhta Takida, Naoki Murata +3
Score-based generative models (SGMs) learn a family of noise-conditional score functions corresponding to the data density perturbed with increasingly large amounts of noise. These…
SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization
Yuhta Takida, Takashi Shibuya, WeiHsiang Liao +7
One noted issue of vector-quantized variational autoencoder (VQ-VAE) is that the learned discrete representation uses only a fraction of the full capacity of the codebook, also kno…
Preventing Oversmoothing in VAE via Generalized Variance Parameterization
Yuhta Takida, Wei-Hsiang Liao, Chieh-Hsin Lai +3
Variational autoencoders (VAEs) often suffer from posterior collapse, which is a phenomenon in which the learned latent space becomes uninformative. This is often related to the hy…