6 papers
Latent State Utilization in Probabilistic Circuits
Zilei Shao, Anji Liu
Probabilistic circuits (PCs) are generative models that support efficient probabilistic inference. Adding latent states to a PC increases its representational capacity, but yields…
ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
Heng Zhao, Zilei Shao, Guy Van den Broeck +1
Mixture-of-Experts (MoE) models scale by activating only a small subset of experts per token. However, training such models remains challenging because top- routing is discrete…
Breaking the Factorization Barrier in Diffusion Language Models
Ian Li, Zilei Shao, Benjie Wang +3
Diffusion language models theoretically allow for efficient parallel generation but are practically hindered by the ``factorization barrier'': the assumption that simultaneously pr…
Zero-Variance Gradients for Variational Autoencoders
Zilei Shao, Anji Liu, Guy Van den Broeck
Training deep generative models like Variational Autoencoders (VAEs) requires propagating gradients through stochastic latent variables, which introduces estimation variance that c…
Adversarial Tokenization
Renato Lui Geh, Zilei Shao, Guy Van den Broeck
Current LLM pipelines account for only one possible tokenization for a given string, ignoring exponentially many alternative tokenizations during training and inference. For exampl…
Bounding-Box Inference for Error-Aware Model-Based Reinforcement Learning
Erin J. Talvitie, Zilei Shao, Huiying Li +4
In model-based reinforcement learning, simulated experiences from the learned model are often treated as equivalent to experience from the real environment. However, when the model…