This paper proposes Hierarchical Cooperative GFlowNets (HCG) to improve exploration in generative flow networks by dynamically partitioning the search space.
This paper introduces a novel approach to coordinating multiple generative flow networks to improve exploration in compositional spaces.
Before reading this…
Applications
To understand this paper, make sure you know these concepts first:
Generative Flow Networks (GFlowNets) are trained to sample objects from compositional spaces with probability proportional to a reward function, but ineffective exploration can cause learned samplers to concentrate on a few high-reward regions. An intuitive remedy is to train multiple GFlowNets independently in parallel and combine their samples. However, this does not prevent multiple models from collapsing onto the same high-reward areas, leaving large regions of the state space unexplored. With this in mind, we propose Hierarchical Cooperative GFlowNets (HCG), an approach that dynamically partitions the search space to coordinate discovery without relying on hand-crafted boundaries. HCG imposes an ordering in which the highest-ranked model may explore the entire space, while lower-ranked models are progressively pushed toward regions not covered by higher-ranked ones. We provide a theoretical characterization of how specific learning objectives can shift probability mass away from designated areas, instantiating this mechanism within our architecture. A shared classifier coordinates the hierarchy by partitioning the search space, keeping the GFlowNets independently parameterized and enabling asynchronous training. Furthermore, we derive a normalizer-weighted aggregation rule for the ensemble and prove that it recovers the target reward-proportional distribution under the stated assumptions. Finally, experiments across multimodal distributions, sequence generation, combinatorial optimization, and biological sequence design show that HCG improves mode coverage and target approximation, translating increases in the number of coordinated samplers into broader, complementary exploration.
PoEM: Predicting RL Outcomes from Existing Policies
This paper introduces PoEM, a framework to predict the outputs of reinforcement…
Decoupling Exploration from Optimization in RLVR
This paper presents a framework called Exploration-Distillation (ExpDis) that de…
Learning Process Rewards via Success Visitation Matching for Efficient RL
This paper proposes a method to transform sparse outcome rewards into dense proc…
Flow-Guided Target-Space Alignment via Path Consistency
This paper proposes a flow-guided target-space alignment method called FlowAlign…
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
One clear sentence describing what this paper does
Self-Evolving Visual Agent via Margin Reward
Marvo is a self-evolution approach that enables learning tool invocation from un…
Hallucination-Aware Multi-Agent RAG via Internal Attention Rewards
A new multi-agent RAG framework that reduces hallucination rates in language mod…
Exploration strategies for Gaussian process bandit optimization with inference regret guarantees
This paper introduces efficient exploration strategies for Bayesian optimization…