Dream to posture: visual posturing of a tendon-driven hand using world model and muscle synergies

摘要

AbstractAlthough tendon-driven anthropomorphic robot hands have the potential to achieve human-level dexterity, controlling them is a great challenge owing to their mechanical complexities. Therefore, investigating human-hand control strategies is of the utmost importance. An important skill that enables the versatile manipulation ability of humans is visual posturing, i.e. the skill to make arbitrary hand postures based solely on visual observation. Visual posturing facilitates manipulation learning by enabling visual imitation learning and reusing visually similar past experiences. Therefore, this study investigates a method to replicate visual posturing in anthropomorphic robotic hands. Visual posturing in tendon-driven hands is challenging because of the hysteresis in tendon systems, the partial observability of the problem, and the presence of many actuators owing to the antagonistic tendon arrangement. To address these challenges, we propose a method that combines a model predictive path integral, a world model, and bio-inspired muscle synergies. The evaluation in a physical tendon-driven anthropomorphic robot hand showed that the proposed method achieved better visual posturing performance than a naive regression model. We anticipate that our visual posturing method will lay the foundation for versatile manipulation controllers that can adaptively learn manipulation tasks, similar to humans.Keywords: Tendon-driven anthropomorphic handvisual controlreinforcement learningworld modelmuscle synergies Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis work was supported by JST ERATO (Grant Number JPMJER1501) and JST Moonshot Research and Developement (Grant Number JPMJMS2033).Notes on contributorsMatthew IshigeMatthew Ishige received his M.Eng. and Ph.D. in Information Communication Engineering from the University of Tokyo in 2019 and 2023, respectively. His research involves learning-based control methods for soft-bodied robots.Tadahiro TaniguchiTadahiro Taniguchi received his M.Eng. and Ph.D. degrees from Kyoto University, in 2003 and 2006, respectively. From 2005 to 2008, he was a Japan Society for the Promotion of Science Research Fellow in the same university. From 2008 to 2010, he was an Assistant Professor at the Department of Human and Computer Intelligence, Ritsumeikan University. From 2010 to 2017, he was an Associate Professor in the same department. From 2015 to 2016, he was a Visiting Associate Professor at the Department of Electrical and Electronic Engineering, Imperial College London. Since 2017, he is a Professor at the Department of Information and Engineering, Ritsumeikan University, and a Visiting General Chief Scientist at the Technology Division of Panasonic Corporation. He has been engaged in research on machine learning, emergent systems, intelligent vehicle and symbol emergence in robotics.Yoshihiro KawaharaYoshihiro Kawahara is currently a professor at the School of Engineering, The University of Tokyo. His research interests are in the areas of Computer Networks and Ubiquitous and Mobile Computing. He is the founder and Director of the Resarch Institute for Inclusive Society through Innovation (RIISE). He also serves as the Head of Research at mercari R4D since 2022. He received Ph.D. in Information Communication Engineering from the University of Tokyo in 2005.