45 citations · 150 across the 25 of their papers we have counts for
27 papers
A Generative Model of Conspicuous Consumption and Status Signaling
Logan Cross, Jordi Grau-Moya, William A. Cunningham +2
Status signaling drives human behavior and the allocation of scarce resources such as mating opportunities, yet the generative mechanisms governing how specific goods, signals, or…
Code World Models for General Game Playing
Wolfgang Lehrach, Daniel Hennes, Miguel Lazaro-Gredilla +13
Large Language Models (LLMs) reasoning abilities are increasingly being applied to classical board and card games, but the dominant approach -- involving prompting for direct move…
Understanding Prompt Tuning and In-Context Learning via Meta-Learning
Tim Genewein, Li Kevin Wenliang, Jordi Grau-Moya +3
Prompting is one of the main ways to adapt a pretrained model to target tasks. Besides manually constructing prompts, many prompt optimization methods have been proposed in the lit…
LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities
Thomas Schmied, Jörg Bornschein, Jordi Grau-Moya +2
The success of Large Language Models (LLMs) has sparked interest in various agentic applications. A key hypothesis is that LLMs, leveraging common sense and Chain-of-Thought (CoT)…
Partition Tree Weighting for Non-Stationary Stochastic Bandits
Joel Veness, Marcus Hutter, Andras Gyorgy +1
This paper considers a generalisation of universal source coding for interaction data, namely data streams that have actions interleaved with observations. Our goal will be to cons…
Why is prompting hard? Understanding prompts on binary sequence predictors
Li Kevin Wenliang, Anian Ruoss, Jordi Grau-Moya +2
Frontier models can be prompted or conditioned to do many tasks, but finding good prompts is not always easy, nor is understanding some performant prompts. We view prompting as fin…
Amortized Planning with Large-Scale Transformers: A Case Study on Chess
Anian Ruoss, Grégoire Delétang, Sourabh Medapati +7
This paper uses chess, a landmark planning problem in AI, to assess transformers' performance on a planning task where memorization is futile $\unicode{x2013}$ even at a large scal…
Learning Universal Predictors
Jordi Grau-Moya, Tim Genewein, Marcus Hutter +8
Meta-learning has emerged as a powerful approach to train neural networks to learn new tasks quickly from limited data. Broad exposure to different tasks leads to versatile represe…
Language Modeling Is Compression
Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne +9
It has long been established that predictive models can be transformed into lossless compressors and vice versa. Incidentally, in recent years, the machine learning community has f…
Randomized Positional Encodings Boost Length Generalization of Transformers
Anian Ruoss, Grégoire Delétang, Tim Genewein +5
Transformers have impressive generalization capabilities on tasks with a fixed context length. However, they fail to generalize to sequences of arbitrary length, even for seemingly…
Memory-Based Meta-Learning on Non-Stationary Distributions
Tim Genewein, Grégoire Delétang, Anian Ruoss +7
Memory-based meta-learning is a technique for approximating Bayes-optimal predictors. Under fairly general conditions, minimizing sequential prediction error, measured by the log l…
Beyond Bayes-optimality: meta-learning what you know you don't know
Jordi Grau-Moya, Grégoire Delétang, Markus Kunesch +11
Meta-training agents with memory has been shown to culminate in Bayes-optimal agents, which casts Bayes-optimality as the implicit solution to a numerical optimization problem rath…
Neural Networks and the Chomsky Hierarchy
Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya +8
Reliable generalization lies at the heart of safe ML and AI. However, understanding when and how neural networks generalize remains one of the most important unsolved problems in t…
Your Policy Regularizer is Secretly an Adversary
Rob Brekelmans, Tim Genewein, Jordi Grau-Moya +4
Policy regularization methods such as maximum entropy regularization are widely used in reinforcement learning to improve the robustness of a learned policy. In this paper, we show…
Model-Free Risk-Sensitive Reinforcement Learning
Grégoire Delétang, Jordi Grau-Moya, Markus Kunesch +4
We extend temporal-difference (TD) learning in order to obtain risk-sensitive, model-free reinforcement learning algorithms. This extension can be regarded as modification of the R…
Shaking the foundations: delusions in sequence models for interaction and control
Pedro A. Ortega, Markus Kunesch, Grégoire Delétang +16
The recent phenomenal success of language models has reinvigorated machine learning research, and large sequence models such as transformers are being applied to a variety of domai…
Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow
John McLeod, Hrvoje Stojic, Vincent Adam +4
In the past decade, model-free reinforcement learning (RL) has provided solutions to challenging domains such as robotics. Model-based RL shows the prospect of being more sample-ef…
Causal Analysis of Agent Behavior for AI Safety
Grégoire Déletang, Jordi Grau-Moya, Miljan Martic +6
As machine learning systems become more powerful they also become increasingly unpredictable and opaque. Yet, finding human-understandable explanations of how they work is essentia…
Mutual-Information Regularization in Markov Decision Processes and Actor-Critic Learning
Felix Leibfried, Jordi Grau-Moya
Cumulative entropy regularization introduces a regulatory signal to the reinforcement learning (RL) problem that encourages policies with high-entropy actions, which is equivalent…
A Unified Bellman Optimality Principle Combining Reward Maximization and Empowerment
Felix Leibfried, Sergio Pascual-Diaz, Jordi Grau-Moya
Empowerment is an information-theoretic method that can be used to intrinsically motivate learning agents. It attempts to maximize an agent's control over the environment by encour…
Disentangled Skill Embeddings for Reinforcement Learning
Janith C. Petangoda, Sergio Pascual-Diaz, Vincent Adam +2
We propose a novel framework for multi-task reinforcement learning (MTRL). Using a variational inference formulation, we learn policies that generalize across both changing dynamic…
Balancing Two-Player Stochastic Games with Soft Q-Learning
Jordi Grau-Moya, Felix Leibfried, Haitham Bou-Ammar
Within the context of video games the notion of perfectly rational agents can be undesirable as it leads to uninteresting situations, where humans face tough adversarial decision m…
An Information-Theoretic Optimality Principle for Deep Reinforcement Learning
Felix Leibfried, Jordi Grau-Moya, Haitham Bou-Ammar
We methodologically address the problem of Q-value overestimation in deep reinforcement learning to handle high-dimensional state spaces efficiently. By adapting concepts from info…
Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes
Jordi Grau-Moya, Felix Leibfried, Tim Genewein +1
Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by…