9 papers
Gaussian Invariant Markov Chain Monte Carlo
Michalis K. Titsias, Angelos Alexopoulos, Siran Liu +1
We develop sampling methods, which consist of Gaussian invariant versions of random walk Metropolis (RWM), Metropolis adjusted Langevin algorithm (MALA) and second order Hessian or…
Variational Learning for Insertion-based Generation
Yangtian Zhang, Zhe Wang, Arthur Gretton +4
Non-monotonic sequence generation methods, such as masked diffusion models, provide a flexible alternative to left-to-right autoregressive modeling by allowing tokens to be generat…
The Illusion of Stochasticity in LLMs
Xiangming Gu, Soham De, Michalis Titsias +3
In this work, we demonstrate that reliable stochastic sampling is a fundamental yet unfulfilled requirement for Large Language Models (LLMs) operating as agents. Agentic systems ar…
Personalized Federated Learning with Exact Stochastic Gradient Descent
Sotirios Nikoloutsopoulos, Iordanis Koutsopoulos, Michalis K. Titsias
We propose a Stochastic Gradient Descent (SGD)-type algorithm for Personalized Federated Learning which can be particularly attractive for mobile energy-limited regimes due to its…
Demystifying Diffusion Objectives: Reweighted Losses are Better Variational Bounds
Jiaxin Shi, Michalis K. Titsias
We derive a new theoretical interpretation of the reweighted losses that are widely used for training diffusion models. Our method is based on constructing a cascade of time-depend…
Learning-Order Autoregressive Models with Application to Molecular Graph Generation
Zhe Wang, Jiaxin Shi, Nicolas Heess +2
Autoregressive models (ARMs) have become the workhorse for sequence generation tasks, since many problems can be modeled as next-token prediction. While there appears to be a natur…