Improved Switching among Temporally Abstract Actions

摘要

Actions Richard S. Sutton Department of Computer Science University of Massachusetts Amherst, MA 01003-4610 rich@cs.umass.edu Satinder Singh Department of Computer Science University of Colorado Boulder, CO 80309-0430 baveja@cs.colorado.edu Doina Precup Department of Computer Science University of Massachusetts Amherst, MA 01003-4610 dprecup@cs.umass.edu Balaraman Ravindran Department of Computer Science University of Massachusetts Amherst, MA 01003-4610 ravi@cs.umass.edu Abstract In robotics and other control applications it is commonplace to have a preexisting set of controllers for solving subtasks, perhaps handcrafted or previously learned or planned, and still face a difficult problem of how to choose and switch among the controllers to solve an overall task as well as possible. In this paper we present a framework based on Markov Decision Processes and Semi-Markov Decision Processes for phrasing this problem, a basic theorem regarding the improvement in performance that ...