6 papers
Agentic Control in Variational Language Models
Yves Ruffenach
We study whether a variational language model can support a minimal and measurable form of agentic control grounded in its own internal evidence. Our model combines local variation…
Variational Neurons in Transformers for Language Modeling
Yves Ruffenach
Transformers for language modeling usually rely on deterministic internal computation, with uncertainty expressed mainly at the output layer. We introduce variational neurons into…
Exploring the Dimensions of a Variational Neuron
Yves Ruffenach
We introduce EVE (Elemental Variational Expanse), a variational distributional neuron formulated as a local probabilistic computational unit with an explicit prior, an amortized po…
Variational Distributional Neuron
Yves Ruffenach
We propose a proof of concept for a variational distributional neuron: a compute unit formulated as a VAE brick, explicitly carrying a prior, an amortized posterior and a local ELB…
Autoregressivity in the Latent Space of a GP-VAE Language Model: An Empirical Ablation Study
Yves Ruffenach
This paper provides an ablation-based analysis of latent autoregression in GP-VAE models, building upon our previous work introducing the architecture. Language models typically re…
Latent-Autoregressive GP-VAE Language Model
Yves Ruffenach
We investigate a fully Latent AutoRegressive scheme based on a Gaussian Process (GP) integrated into a Variational Autoencoder (VAE). In this setting, sequential dynamics are trans…