1 citations · 1 across the 4 of their papers we have counts for
5 papers
On Reparameterizing the Score Function
Mohammad Hossein Amani, Leonardo Bocchieri, Maxime Peyrard +2
Policy gradients for autoregressive sequence models differentiate each token's log-probability while holding the conditioning history fixed. This discards information about how ear…
Cross-Entropy Games and Frost Training
Arthur Renard, Franck Gabriel, Valentin Hartmann +1
We present Frost Training, a method for improving Monte Carlo-based policy optimization for a large family of LLM-as-a-judge tasks called Cross-Entropy Games. The key idea is to ex…
Cognitive Training for Language Models: Towards General Capabilities via Cross-Entropy Games
Clément Hongler, Franck Gabriel, Valentin Hartmann +2
Defining a constructive process to build general capabilities for language models in an automatic manner is considered an open problem in artificial intelligence. Towards this, we…
Looking for Complexity at Phase Boundaries in Continuous Cellular Automata
Vassilis Papadopoulos, Guilhem Doat, Arthur Renard +1
One key challenge in Artificial Life is designing systems that display an emergence of complex behaviors. Many such systems depend on a high-dimensional parameter space, only a sma…
Arrows of Time for Large Language Models
Vassilis Papadopoulos, Jérémie Wenger, Clément Hongler
We study the probabilistic modeling performed by Autoregressive Large Language Models (LLMs) through the angle of time directionality, addressing a question first raised in (Shanno…