A Theory of Non-Acyclic Generative Flow Networks

Abstract

GFlowNets is a novel flow-based method for learning a stochastic policy to generate objects via a sequence of actions and with probability proportional to a given positive reward. We contribute to relaxing hypotheses limiting the application range of GFlowNets, in particular: acyclicity (or lack thereof). To this end, we extend the theory of GFlowNets on measurable spaces which includes continuous state spaces without cycle restrictions, and provide a generalization of cycles in this generalized context. We show that losses used so far push flows to get stuck into cycles and we define a family of losses solving this issue. Experiments on graphs and continuous tasks validate those principles.

arXiv comment16 pages, 8 figures, 1 table, AAAI 2024

References in corpus (19)

• MCMC using Hamiltonian dynamics
• OpenAI Gym
• A Generalization of Transformer Networks to Graphs
• A Distributional Perspective on Reinforcement Learning
• Biological Sequence Design with GFlowNets
• A theory of continuous generative flow networks
• CFlowNets: Continuous Control with Generative Flow Networks
• Learning GFlowNets from partial episodes for improved convergence and stability
• Unifying Generative Models with GFlowNets and Beyond
• Neural Topological Ordering for Computation Graphs
• Local-Global MCMC kernels: the best of both worlds
• Multi-Objective GFlowNets
• DAG Matters! GFlowNets Enhanced Explainer For Graph Neural Networks
• GFlowNets with Human Feedback
• Stochastic Generative Flow Networks
• Robust Scheduling with GFlowNets
• Generative Flow Networks: a Markov Chain Perspective
• Generative Flow Networks for Precise Reward-Oriented Active Learning on Graphs
• Generalized Universal Domain Adaptation with Generative Flow Networks