Planning with Partially Specified Behaviors

Abstract

In this paper we present a framework called Planning with Partially Specified Behaviors, or PPSB, for combining reinforcement learning and planning to solve sequential decision problems. Although not often combined, we show that reinforcement learning and planning complement each other well, in that each can take advantage of the strengths of the other. PPSB uses partial action specifications to decompose sequential decision problems into tasks that serve as an interface between reinforcement learning and planning. On the bottom level, we use reinforcement learning to compute policies for achieving each individual task. On the top level, we use planning to produce a sequence of tasks that achieves an overall goal. We validate PPSB in experiments in which a robot has to perform tasks in a realistic simulated environment.