Human–Robot Collaboration in 3D via Force Myography Based Interactive Force Estimations Using Cross-Domain Generalization

摘要

In this study, human robot collaboration (HRC) via force myography (FMG) bio-signal was investigated. Interactive hand force was estimated during moving a wooden rod in 3D with a Kuka robot. A baseline FMG-based deep convolutional neural network (FMG-DCNN) model could moderately estimate applied forces during the HRC task. Model performance can be improved with additional training data; however, collection of it was impractical and time-consuming. Available long-term multiple source data (32 feature spaces) during human robot interaction (HRI) with a linear robot collected over a long time period might be useful. Therefore, we explored a cross-domain generalization (CDG) technique that allowed pretraining a model to transfer knowledge between two unrelated source (2D-HRI) and target data (3D-HRC) for the first time. An FMG-based transfer learning with CDG (TL-CDG) model trained with these multiple source domains was examined in estimating applied forces from 16-channel FMG data during interactions with the Kuka robot. Two target scenarios were evaluated: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">case</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$i$ </tex-math></inline-formula> ) collaborative task of moving the wooden rod in 3D, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">case ii)</i> grasping interactions in 1D. In both cases, few calibration data finetuned the TL-CDG model and improved recognizing out-of-domain target data (case i: <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{R}^{2}\approx 60$ </tex-math></inline-formula> -63%, and case ii: <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{R}^{2}\approx 79$ </tex-math></inline-formula> -87%) compared to the baseline FMG-DCNN model. Hence, cross-domain generalization could be useful in platform-independent FMG-based HRI applications.