Pose Estimation-Based Visual Perception System for Analyzing Fish Swimming

摘要

Advances in modern deep learning-based computer vision perception techniques have revolutionized animal movement research methods. These techniques have also opened up new avenues for studying fish swimming. To that end, we have developed a visual perception system based on pose estimation to analyze fish swimming. Our system can quantify fish motion by 3-D fish pose estimation and dynamically visualize the motion data of marked keypoints. Our experimental results show that our system can accurately extract the motion characteristics of fish swimming, which analyze how fish bodies and fins work together during different swimming states. This research provides an innovative idea for studying fish swimming, which can be valuable in designing, developing, and optimizing modern underwater robots, especially multifin codriven bionic robotic fish. The code and dataset are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/wux024/AdamPosePlug</uri> .