Papers
47
Total Citations
2,529
H-Index
21
About
Nicolas Heess is a prominent researcher at the intersection of deep reinforcement learning, robotics, and motor control, whose work has fundamentally advanced how autonomous agents acquire complex physical skills. Based primarily at DeepMind, Heess has made seminal contributions to sparse-reward reinforcement learning, most notably through frameworks that integrate demonstration data with policy gradient methods — work that has collectively garnered hundreds of citations and shaped modern robot learning pipelines. His landmark contributions include leveraging demonstrations for sparse-reward robotics (510 citations), the Scheduled Auxiliary Control paradigm for learning from scratch (155 citations), and the widely adopted dm_control simulation suite (186 citations), which has become a standard benchmark platform for the continuous control community. Heess has also pioneered sim-to-real transfer techniques and skill embedding approaches that enable robots to generalize across diverse manipulation and locomotion tasks. His more recent work training bipedal humanoid robots to play soccer using deep RL (147 citations) exemplifies his ambition to bridge fundamental research and physically embodied intelligence. Spanning visuomotor control, imitation learning, and whole-body animation, his research continues to define the frontier of what autonomous physical agents can achieve.
Research Focus
Key Achievements
Top Papers
- 1
- 2Reinforcement and Imitation Learning for Diverse Visuomotor Skills217 citations · 2018
- 3Learning an Embedding Space for Transferable Robot Skills190 citations · 2018
- 4dm_control: Software and tasks for continuous control186 citations · 2020
- 5Learning by Playing - Solving Sparse Reward Tasks from Scratch155 citations · 2018
- 6
- 7Data-efficient Deep Reinforcement Learning for Dexterous Manipulation118 citations · 2017
- 8Reinforcement and Imitation Learning for Diverse Visuomotor Skills116 citations · 2018
- 9Sim-to-Real Robot Learning from Pixels with Progressive Nets109 citations · 2016
- 10Catch & Carry98 citations · 2020