Non-Markovian policies in sequential decision problems

Abstract

In this article we prove the validity of the Dellman Optimality Equation a.nd related results for sequential decision problems with a general recursive structure. The characteristic feature of our approach is that also non-Markovian policies are taken into account. The theory is motivated The theory of sequential decision problems is an important mathematical tool for studying some problems of cybernetics, e.g. control of robots. Consider for example the robot shmvn in Figure 1. This robot, called Khepera1, is equipped with eight infra-red sensors, six in the front and two at the back, the infra-red