58 citations · 216 across the 28 of their papers we have counts for
30 papers
Counterfactual Probing for Parallel Unmasking with Hidden Forest Structure
Ryotaro Kawata, Satoshi Hayakawa, Taiji Suzuki
Masked generative models offer parallel token prediction, but accurate parallel sampling must account for dependencies among tokens. When dependencies are unknown, finding safe bat…
Does Uniform Discrete Diffusion Need Time?
Chunsan Hong, Chieh-Hsin Lai, Satoshi Hayakawa +3
Uniform discrete diffusion models (UDMs) commonly use explicit time conditioning, but we find that it can often be unnecessary in practice. In this paper, we first show that the po…
From Truncation to Commitment: Persistent Context in Uniform Discrete Diffusion
Satoshi Hayakawa
Uniform-state discrete diffusion models update all tokens in parallel while keeping every position revisable. Even when the commonly used top- rule leaves only one candidate at…
Simplex Relaxation for Discrete Diffusion
Jinya Sakurai, Patrick Pynadath, Satoshi Hayakawa +4
Discrete diffusion models for categorical generation are defined by a corruption kernel, which determines the intermediate state space and the associated reverse prediction problem…
Convex-Geometric Error Bounds for Positive-Weight Kernel Quadrature
Satoshi Hayakawa
Kernel quadrature can exploit RKHS spectral structure and outperform Monte Carlo on smooth integrands, but optimized quadrature weights are generally signed and may be numerically…
Understanding and Accelerating the Training of Masked Diffusion Language Models
Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai +5
Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling. However, MDMs are known to learn substantially more sl…
Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusion
Satoshi Hayakawa, Yuhta Takida, Masaaki Imaizumi +2
Masked diffusion models have shown promising performance in generating high-quality samples in a wide range of domains, but accelerating their sampling process remains relatively u…
MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval
Qiyu Wu, Shuyang Cui, Satoshi Hayakawa +3
Multimodal retrieval, which seeks to retrieve relevant content across modalities such as text or image, supports applications from AI search to contents production. Despite the suc…
Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity
Naoki Yoshida, Satoshi Hayakawa, Yuhta Takida +3
In this study, we propose an enhancement to the similarity computation mechanism in multimodal contrastive pretraining frameworks such as CLIP. Prior theoretical research has demon…
SONA: Learning Conditional, Unconditional, and Mismatching-Aware Discriminator
Yuhta Takida, Satoshi Hayakawa, Takashi Shibuya +6
Deep generative models have made significant advances in generating complex content, yet conditional generation remains a fundamental challenge. Existing conditional generative adv…
Concept-TRAK: Understanding how diffusion models learn concepts through concept-level attribution
Yonghyun Park, Chieh-Hsin Lai, Satoshi Hayakawa +7
While diffusion models excel at image generation, their growing adoption raises critical concerns about copyright issues and model transparency. Existing attribution methods identi…
: Optimizing Sampling Schedule of Discrete Diffusion Models
Yong-Hyun Park, Chieh-Hsin Lai, Satoshi Hayakawa +2
Diffusion models have seen notable success in continuous domains, leading to the development of discrete diffusion models (DDMs) for discrete variables. Despite recent advances, DD…
Distillation of Discrete Diffusion through Dimensional Correlations
Satoshi Hayakawa, Yuhta Takida, Masaaki Imaizumi +2
Diffusion models have demonstrated exceptional performances in various fields of generative modeling, but suffer from slow sampling speed due to their iterative nature. While this…
A Quadrature Approach for General-Purpose Batch Bayesian Optimization via Probabilistic Lifting
Masaki Adachi, Satoshi Hayakawa, Martin Jørgensen +3
Parallelisation in Bayesian optimisation is a common strategy but faces several challenges: the need for flexibility in acquisition functions and kernel choices, flexibility dealin…
Policy Gradient with Kernel Quadrature
Satoshi Hayakawa, Tetsuro Morimura
Reward evaluation of episodes becomes a bottleneck in a broad range of reinforcement learning tasks. Our aim in this paper is to select a small but representative subset of a large…
Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approach
Masaki Adachi, Satoshi Hayakawa, Martin Jørgensen +4
Active learning parallelization is widely used, but typically relies on fixing the batch size throughout experimentation. This fixed approach is inefficient because of a dynamic tr…
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation
Hayata Yamasaki, Sathyawageeswar Subramanian, Satoshi Hayakawa +1
A significant challenge in the field of quantum machine learning (QML) is to establish applications of quantum computation to accelerate common tasks in machine learning such as th…
SOBER: Highly Parallel Bayesian Optimization and Bayesian Quadrature over Discrete and Mixed Spaces
Masaki Adachi, Satoshi Hayakawa, Saad Hamid +3
Batch Bayesian optimisation and Bayesian quadrature have been shown to be sample-efficient methods of performing optimisation and quadrature where expensive-to-evaluate objective f…
Sampling-based Nyström Approximation and Kernel Quadrature
Satoshi Hayakawa, Harald Oberhauser, Terry Lyons
We analyze the Nyström approximation of a positive definite kernel associated with a probability measure. We first prove an improved error bound for the conventional Nyström approx…
Hypercontractivity Meets Random Convex Hulls: Analysis of Randomized Multivariate Cubatures
Satoshi Hayakawa, Harald Oberhauser, Terry Lyons
Given a probability measure on a set and a vector-valued function , a common problem is to construct a discrete probability measure on such that…
Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombination
Masaki Adachi, Satoshi Hayakawa, Martin Jørgensen +2
Calculation of Bayesian posteriors and model evidences typically requires numerical integration. Bayesian quadrature (BQ), a surrogate-model-based approach to numerical integration…
Positively Weighted Kernel Quadrature via Subsampling
Satoshi Hayakawa, Harald Oberhauser, Terry Lyons
We study kernel quadrature rules with convex weights. Our approach combines the spectral properties of the kernel with recombination results about point measures. This results in e…
Estimating the probability that a given vector is in the convex hull of a random sample
Satoshi Hayakawa, Terry Lyons, Harald Oberhauser
For a -dimensional random vector , let be the probability that the convex hull of independent copies of contains a given point . We provide several s…
Monte Carlo construction of cubature on Wiener space
Satoshi Hayakawa, Ken'ichiro Tanaka
In this paper, we investigate application of mathematical optimization to construction of a cubature formula on Wiener space, which is a weak approximation method of stochastic dif…