Reward Prediction Error Signals are Meta‐Representational

摘要

Contents 1. Introduction 2. Reward-Guided Decision Making 3. Content in the Model 4. How to Deflate a Metarepresentational Reading Proust and Carruthers on metacognitive feelings 5. A Deflationary Treatment of RPEs? 5.1 Dispensing with prediction errors 5.2 What is use of the RPE focused on? 5.3 Alternative explanations—worldly correlates 5.4 Contrast cases 6. Conclusion Appendix: Temporal Difference Learning Algorithms It is often thought that metarepresentation is a particularly sophisticated cognitive achievement. There is substantial evidence that dolphins and some primates can represent some of their own psychological states, but the existence of ‘metacognition’ in any other species remains highly controversial (Carruthers 2009; Hampton 2009; Smith 2009). Research on the extent to which humans metacognize their own psychological states is expanding in parallel (Fleming et al. forthcoming). Representing the mental states of others is believed to be even rarer or non-existent in non-human animals (Hare et al. 2001; Heyes 1998). That capacity, under the label ‘theory of mind’, was long thought to be a crucial and relatively late developmental transition even in humans (Leslie 1987; Perner et al. 1989; Wimmer and Perner 1983). Recent looking-time experiments suggesting infants have an ability to track others’ perceptions and beliefs at a very young age raise the possibility that infants have some lower-level capacity for mentalizing (Kovács et al. 2010; Onishi and Baillargeon 2005; Surian et al. 2007), in which case they may be able to metarepresent before they have a fully-fledged concept of belief, desire, or any other psychological state (Apperly and Butterfill 2009). However, macaques do not show the same behaviour (Marticorena et al. 2011) and there has been no suggestion that the capacity for non-conceptual metarepresentation extends more widely than that. This paper argues that non-conceptual metarepresentation does extend much more widely, but based on a different set of considerations, located in a field where the issue of metarepresentation has been overlooked: the literature on reinforcement learning in reward-guided decision-making tasks. Research on humans and other animals has produced an impressive body of converging evidence that midbrain dopamine neurons produce a prediction that is in behaviour et al. 2009; et al. 1998). in and even et al. 2009). This paper argues that A metarepresentation is a the of the of a non-conceptual is a It that the use of a non-conceptual does not the of any non-conceptual there is no suggestion that a of or of mental a more of and different the of the mental states of and others that is It is that the does of the that in the of non-conceptual has been in some for and a non-conceptual represent a It is to to of a A thought a the thought the concept and of a metarepresentation to represent a have a that to the of a some other of a belief, a or to some That is non-conceptual However, there is in the of non-conceptual that in the or of a non-conceptual for non-conceptual can in the there a of the can a in which a non-conceptual to represent a represent of the of is more than or the of the of is the in the is a is but does not represent a no of in the of the other the of the of for the of the of the is to be represent a the in which non-conceptual of the a or that the of is to be a for to be a That is a It is not that A in or This paper argues that non-conceptual is for the reward-guided decision-making that is in and that of the RPE to have non-conceptual the which a for reinforcement learning of behaviour is that a of non-conceptual metarepresentation is relatively in the the does not that metarepresentation is It is of of the and that an for metarepresentation can be is that the and to produce evidence for not is which that a may to other in which the a prediction and is to the prediction for the 2010; et al. However, the does not to to for of and is the to the on the behaviour in i