6 papers
Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
Kareem Ahmed, Sameer Singh
Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generati…
Scaling Tractable Probabilistic Circuits: A Systems Perspective
Anji Liu, Kareem Ahmed, Guy Van den Broeck
Probabilistic Circuits (PCs) are a general framework for tractable deep generative models, which support exact and efficient probabilistic inference on their learned distributions.…
Where is the signal in tokenization space?
Renato Lui Geh, Honghua Zhang, Kareem Ahmed +2
Large Language Models (LLMs) are typically shipped with tokenizers that deterministically encode text into so-called canonical token sequences, to which the LLMs assign probability…
Controllable Generation via Locally Constrained Resampling
Kareem Ahmed, Kai-Wei Chang, Guy Van den Broeck
Autoregressive models have demonstrated an unprecedented ability at modeling the intricacies of natural language. However, they continue to struggle with generating complex outputs…
SIMPLE: A Gradient Estimator for -Subset Sampling
Kareem Ahmed, Zhe Zeng, Mathias Niepert +1
-subset sampling is ubiquitous in machine learning, enabling regularization and interpretability through sparsity. The challenge lies in rendering -subset sampling amenable t…
Semantic Loss Functions for Neuro-Symbolic Structured Prediction
Kareem Ahmed, Stefano Teso, Paolo Morettin +8
Structured output prediction problems are ubiquitous in machine learning. The prominent approach leverages neural networks as powerful feature extractors, otherwise assuming the in…