cs.ROcs.LGEmpirical

Hierarchical Soft Actor-Critic for Sparse-Reward Long-Horizon Reinforcement Learning

Zahra Abdalla Elashaal, Afef Hfaiedh, Nahla Khraief, Issmail Ellabib, Giansalvo Cirrincione

Jul 26, 2026

AI Summaryllama-3.1-8b-instruct

The paper proposes a Hierarchical Reinforcement Learning framework with two levels for handling high-level strategic planning and low-level continuous-control using Soft Actor-Critic and entropy-regularized policy optimization. It outperforms flat SAC in sparse-reward long-horizon search tasks.

Proposed a Hierarchical Reinforcement Learning framework for sparse-reward long-horizon tasks

Keywords

Reinforcement Learning Hierarchical Reinforcement Learning Soft Actor-Critic Sparse-Reward Long-Horizon

Before reading this…

Understanding of Reinforcement Learning

Applications

→Search-and-Rescue tasks

Skill Ladder

To understand this paper, make sure you know these concepts first:

Understanding of Reinforcement Learningfind papers →

Abstract

More Like This

Exploration in sparse-reward long-horizon tasks poses significant challenges for reinforcement learning. To address these challenges, we propose a two-level Hierarchical Reinforcement Learning (HRL) framework. The first level handles high-level strategic planning, while the low-level uses the continuous-control Soft Actor-Critic (SAC) algorithm, and they utilize entropy-regularized policy optimization. The proposed framework was trained and evaluated using the Search-and-Rescue-2 (SAR-2) dataset. HRL-SAC effectively addresses sparse-reward long-horizon search problems characterized by delayed rewards and continuous control, and its outperforming the flat SAC baseline reinforcement learning in terms of success rates, coverage efficiency, and convergence. These findings indicate that hierarchical entropy-regularized policies are a promising solution to tackle long-horizon sparse-reward reinforcement learning tasks.