Deep Learning-Assisted 3D Pressure Sensors for Control of Unmanned Aerial Vehicles

摘要

Accurately and reliably detecting and recognizing human body movements in real time, relaying appropriate commands to the machine, have substantial implications for virtual reality, remote control, and robotics applications. Nonetheless, most contemporary wearable analysis and control systems attain action recognition by setting sensor thresholds. In routine usage, the stringent trigger conditions facilitate inadvertent contact, resulting in a poorer user experience. Here, we have created a wearable intelligent gesture recognition control system utilizing a multilayer microstructure composite thin film piezoresistive sensing array and deep learning techniques. The system exhibits ultrahigh sensitivity (ranging from 0–6 kPa to 412.2 kPa–1) and rapid response times (loading at 40 ms, recovery at 30 ms). The detected gestures are classified and recognized via a convolutional neural network, achieving a recognition accuracy of 97.5%. Ultimately, the altitude control of an unmanned aerial vehicle is accomplished through wireless signal transmission and reception. To achieve the visualization of the complete gesture-controlled flight process, we developed an intuitive user interface for the real-time display of flight altitude and video surveillance. The implementation of this recognition system introduces a novel control mechanism for human–machine interaction, expands the applications of robotic technology, and offers innovative concepts and practical pathways for virtual reality.