5 papers
Mitigating Bias in Locally Constrained Decoding via Tractable Proposals
Meihua Dang, Linxin Song, Honghua Zhang +3
Generations from large language models often fail to conform to desired constraints such as JSON schema. Existing locally constrained decoding (LCD) approaches enforce constraints…
Scaling Probabilistic Circuits via Monarch Matrices
Honghua Zhang, Meihua Dang, Benjie Wang +3
Probabilistic Circuits (PCs) are tractable representations of probability distributions allowing for exact and efficient computation of likelihoods and marginals. Recent advancemen…
Where is the signal in tokenization space?
Renato Lui Geh, Honghua Zhang, Kareem Ahmed +2
Large Language Models (LLMs) are typically shipped with tokenizers that deterministically encode text into so-called canonical token sequences, to which the LLMs assign probability…
Restructuring Tractable Probabilistic Circuits
Honghua Zhang, Benjie Wang, Marcelo Arenas +1
Probabilistic circuits (PCs) are a unifying representation for probabilistic models that support tractable inference. Numerous applications of PCs like controllable text generation…
Scaling Up Probabilistic Circuits by Latent Variable Distillation
Anji Liu, Honghua Zhang, Guy Van den Broeck
Probabilistic Circuits (PCs) are a unified framework for tractable probabilistic models that support efficient computation of various probabilistic queries (e.g., marginal probabil…