Editorial: AI-powered musical and entertainment robotics

Abstract

The convergence of robotics and artificial intelligence (AI) is revolutionizing the field of music and entertainment. Robots are evolving from performing traditional service-oriented tasks to enabling advanced human-robot interaction (HRI) with potential emotional engagement. The pursuit of robotic expressiveness presents new challenges and opportunities in the modeling, design and control of musical and entertainment robots. Current studies mainly work on the design and physical implementation of robots capable of manipulating various musical instruments \cite{wang2022data,lim2012towards}, while the development of socially intelligent robots for real-time HRI remains underexplored. With advancements in AI, robots can now compose and improvise, as well as interpret and respond to human affective states during HRI \cite{mccoll2016survey, wang2024human}.This research topic was initiated to present the latest developments of AI-powered musical and entertainment robots. As a result of the call, six papers have been accepted and collected in this research topic. These articles provide a comprehensive exploration of diverse artistic forms including singing, dancing and musical performance on instruments such as the piano, violin, guitar, drum and marimba. Figure \ref{fig:overview} shows an overview of the musical robots investigated in these studies.Among the contributed works, two articles focused on dexterous manipulation and sensorimotor coordination. \href{https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2024.1463744/full}{Gilday et al.} introduced a general-purpose system featuring a parametric hand capable of playing both the piano and performing guitar pick strumming. Unlike existing bespoke robotic musical systems, the proposed hand was designed as a single-piece 3D-printed structure, demonstrating potential for enhanced expressiveness in entertainment applications through the modulation of mechanical properties and actuation modes. The study highlighted that leveraging system-environment interactions enabled diverse, multi-instrument functionalities and variable playing styles with simplified control. Instead of musical instrument playing, \href{https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2024.1450177/full}{Twomey et al.} investigated dance performance using wearable soft sensors on the arm to explore whether such devices could enhance artistic expression. Dance movements were modeled as colliders within virtual mass-spring-damper systems, and limb segments were analyzed in local frames to avoid drift issues commonly associated with IMUs. The authors proposed a parallel algorithm to detect improvisational dance movements and control soft wearable actuators which can change size and lighting in response to detected motions. This work exemplified sensorimotor coordination and demonstrated how traditional dance and aesthetics could be enriched by spontaneous wearable-driven movements.Robot learning and control represent one of the biggest challenges in musical and entertainment robotics, particularly for acquiring manipulation skills and robotic expressiveness. \href{https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2024.1439629/full}{Horigome and Shibuya} developed a RL-based controller for a violin-playing robot, a 7-DoF dual-arm system actuated by DC motors. The system mimics human performance with the left arm handling fingering and the right arm controlling bowing movements. The right arm regulates multiple parameters including bowing speed, pressure, sounding point and direction. Analysis of the target sound pressure demonstrated that the robot successfully learned violin-playing techniques and enables expressive performance variations. The robot was automated to play the violin based on musical scores, demonstrating its ability to interpret and execute complex musical tasks. Similarly, \href{https://www.frontiersin.org/journals/robotics