Towards Leveraging Edge Intelligence in Tactile Systems: Enhancing Grasping of Soft Objects Through Haptic Feedback

Abstract

Remote tactile sensing necessitates communication of multi-modal data such as audio-visual detailing in addition to the haptic feedback required to characterize a sense of touch observed during robotic teleoperation. We consider a scenario where a deformable 3D object is manipulated by a robotic gripper by a remotely operated dexterous exoskeleton. Owing to the time-sensitive nature of the activity, the remoteness of the operator can lead to unwanted scenarios that need to be prevented, such as excessive deformation of the object by the gripper. We propose an edge intelligence-based split-control mechanism for softness-adaptive grasping with haptic feedback. The proposed approach utilizes a Masked Region Convolutional Neural Network (RCNN) based approach, deployed at the edge, to detail the deformation characteristics of the soft object being grasped. The edge controller is trained on a custom dataset and adapts the gripper position when deformation goes beyond a threshold level based on the change of segment instances count of the soft body. We focus on analyzing the scope of edge intelligence in tactile applications to achieve softness-adaptive grasping of deformable objects while enhancing the grasping experience through haptic feedback based on kinesthetic sampling. We proceed to study the feasibility of edge analytics in preventing such events from occurring.