Hindsight Experience Replay

ABSTRACT

Dealing with sparse rewards is one of the biggest challenges in Reinforcement Learning (RL). We present a novel technique called Hindsight Experience Replay (HER) which allows sample-efficient learning from rewards which are sparse and binary. HER can be combined with any off-policy RL algorithm and is applicable whenever there are multiple goals that can be achieved. The core idea is to replay past experiences with the goals that were actually achieved, rather than the original intended goals. This allows the agent to learn from failure, recognizing that even if it failed its intended task, it successfully learned how to achieve the state it ended up in.