19 citations · 46 across the 22 of their papers we have counts for
23 papers
COrigami: An AI Pipeline for Co-Designing Flat-Foldable Visually Recognisable Origami
Tom Zahavy, Shaobo Hou, Thomas Tumiel +16
While generative AI has achieved remarkable success in solving problems with verifiable solutions, generating physical art that satisfies both strict geometric constraints and subj…
Curious Causality-Seeking Agents Learn Meta Causal World
Zhiyu Zhao, Haoxuan Li, Haifeng Zhang +4
When building a world model, a common assumption is that the environment has a single, unchanging underlying causal rule, like applying Newton's laws to every situation. In reality…
On the Convergence and Stability of Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning, and Online Decision Transformers
Miroslav Štrupl, Oleg Szehr, Francesco Faccio +3
This article provides a rigorous analysis of convergence and stability of Episodic Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning and Online Decision Tran…
Upside Down Reinforcement Learning with Policy Generators
Jacopo Di Ventura, Dylan R. Ashley, Vincent Herrmann +2
Upside Down Reinforcement Learning (UDRL) is a promising framework for solving reinforcement learning problems which focuses on learning command-conditioned policies. In this work,…
How to Correctly do Semantic Backpropagation on Language-based Agentic Systems
Wenyi Wang, Hisham A. Alyahya, Dylan R. Ashley +4
Language-based agentic systems have shown great promise in recent years, transitioning from solving small-scale research problems to being deployed in challenging real-world tasks.…
Highway Value Iteration Networks
Yuhui Wang, Weida Li, Francesco Faccio +2
Value iteration networks (VINs) enable end-to-end learning for planning tasks by employing a differentiable "planning module" that approximates the value iteration algorithm. Howev…
Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning
Yuhui Wang, Qingyuan Wu, Dylan R. Ashley +4
The Value Iteration Network (VIN) is an end-to-end differentiable neural network architecture for planning. It exhibits strong generalization to unseen domains by incorporating a d…
Highway Reinforcement Learning
Yuhui Wang, Miroslav Strupl, Francesco Faccio +5
Learning from multi-step off-policy data collected by a set of policies is a core problem of reinforcement learning (RL). Approaches based on importance sampling (IS) often suffer…
Towards a Robust Soft Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms
Mohannad Alhakami, Dylan R. Ashley, Joel Dunham +4
Advanced machine learning algorithms require platforms that are extremely robust and equipped with rich sensory feedback to handle extensive trial-and-error learning without relyin…
Faster Diffusion via Temporal Attention Decomposition
Haozhe Liu, Wentian Zhang, Jinheng Xie +6
We explore the role of attention mechanism during inference in text-conditional diffusion models. Empirical observations suggest that cross-attention outputs converge to a fixed po…
Learning Useful Representations of Recurrent Neural Network Weight Matrices
Vincent Herrmann, Francesco Faccio, Jürgen Schmidhuber
Recurrent Neural Networks (RNNs) are general-purpose parallel-sequential computers. The program of an RNN is its weight matrix. How to learn useful representations of RNN weights t…
Language Agents as Optimizable Graphs
Mingchen Zhuge, Wenyi Wang, Louis Kirsch +3
Various human-designed prompt engineering techniques have been proposed to improve problem solvers based on Large Language Models (LLMs), yielding many disparate code bases. We uni…
The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute
Aleksandar Stanić, Dylan Ashley, Oleg Serikov +5
The Languini Kitchen serves as both a research collective and codebase designed to empower researchers with limited computational resources to contribute meaningfully to the field…
Learning to Identify Critical States for Reinforcement Learning from Videos
Haozhe Liu, Mingchen Zhuge, Bing Li +4
Recent work on deep reinforcement learning (DRL) has pointed out that algorithmic information about good policies can be extracted from offline data which lack explicit information…
Mindstorms in Natural Language-Based Societies of Mind
Mingchen Zhuge, Haozhe Liu, Francesco Faccio +23
Both Minsky's "society of mind" and Schmidhuber's "learning to think" inspire diverse societies of large multimodal neural networks (NNs) that solve problems by interviewing each o…
Goal-Conditioned Generators of Deep Policies
Francesco Faccio, Vincent Herrmann, Aditya Ramesh +2
Goal-conditioned Reinforcement Learning (RL) aims at learning optimal policies, given goals encoded in special command inputs. Here we study goal-conditioned neural nets (NNs) that…
General Policy Evaluation and Improvement by Learning to Identify Few But Crucial States
Francesco Faccio, Aditya Ramesh, Vincent Herrmann +2
Learning to evaluate and improve policies is a core problem of Reinforcement Learning (RL). Traditional RL algorithms learn a value function defined for a single policy. A recently…
Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules
Kazuki Irie, Francesco Faccio, Jürgen Schmidhuber
Neural ordinary differential equations (ODEs) have attracted much attention as continuous-time counterparts of deep residual neural networks (NNs), and numerous extensions for recu…
Upside-Down Reinforcement Learning Can Diverge in Stochastic Environments With Episodic Resets
Miroslav Štrupl, Francesco Faccio, Dylan R. Ashley +2
Upside-Down Reinforcement Learning (UDRL) is an approach for solving RL problems that does not require value functions and uses only supervised learning, where the targets for give…
Bayesian brains and the Rényi divergence
Noor Sajid, Francesco Faccio, Lancelot Da Costa +3
Under the Bayesian brain hypothesis, behavioural variations can be attributed to different priors over generative model parameters. This provides a formal explanation for why indiv…
Reward-Weighted Regression Converges to a Global Optimum
Miroslav Štrupl, Francesco Faccio, Dylan R. Ashley +2
Reward-Weighted Regression (RWR) belongs to a family of widely known iterative Reinforcement Learning algorithms based on the Expectation-Maximization framework. In this family, le…
Parameter-Based Value Functions
Francesco Faccio, Louis Kirsch, Jürgen Schmidhuber
Traditional off-policy actor-critic Reinforcement Learning (RL) algorithms learn value functions of a single target policy. However, when value functions are updated to track the l…
Policy Optimization via Importance Sampling
Alberto Maria Metelli, Matteo Papini, Francesco Faccio +1
Policy optimization is an effective reinforcement learning approach to solve continuous control tasks. Recent achievements have shown that alternating online and offline optimizati…