7 papers
World models of environment, agent and joint agent-environment systems
Manuel Baltieri, Filippo Torresan, Yivan Zhang +2
World models are a central component of model-based reinforcement learning. They are usually discussed in terms of what variables they predict, such as observations, rewards, state…
Bayesian updates from coalgebraic determinisation
Manuel Baltieri, Nathaniel Virgo
The powerset construction is the classical determinisation procedure for nondeterministic finite automata. In the coalgebraic setting, this construction has been generalised to str…
Compositional Behavioral Semantics for State Abstraction in Reinforcement Learning
Yivan Zhang, Ziyan Luo, Manuel Baltieri
State abstraction plays a key role in scaling reinforcement learning to complex but structured systems. In studying such systems, a wide range of behavioral structures have been st…
Prior preferences in active inference agents: soft, hard, and goal shaping
Filippo Torresan, Ryota Kanai, Manuel Baltieri
Active inference proposes expected free energy as an objective for planning and decision-making to adequately balance exploitative and explorative drives in learning agents. The ex…
A coalgebraic perspective on predictive processing
Manuel Baltieri, Filippo Torresan, Tomoya Nakai
Predictive processing and active inference posit that the brain is a system performing Bayesian inference on the environment. By virtue of this, a prominent interpretation of predi…
A "good regulator theorem" for embodied agents
Nathaniel Virgo, Martin Biehl, Manuel Baltieri +1
In a classic paper, Conant and Ashby claimed that "every good regulator of a system must be a model of that system." Artificial Life has produced many examples of systems that perf…