Policy-Gradient Reinforcement Learning as a General Theory of Practice-Based Motor Skill Learning

摘要

Abstract Mastering any new skill requires extensive practice, but the computational principles underlying this learning are not clearly understood. Existing theories of motor learning can explain short-term adaptation to perturbations, but offer little insight into the processes that drive gradual skill improvement through practice. Here, we propose that practice-based motor skill learning can be understood as a form of reinforcement learning (RL), specifically, policy-gradient RL, a simple, model-free method that is widely used in robotics and other continuous control settings. Here, we show that models based on policy-gradient learning rules capture key properties of human skill learning across a diverse range of learning tasks that have previously lacked any computational theory. We suggest that policy-gradient RL can provide a general theoretical framework and foundation for understanding how humans hone skills through practice.