Towards Automated Tissue Classification for Markerless Orthopaedic Robotic Assistance

Abstract

A markerless computer aided orthopaedic platform will require a complex computer vision system to isolate and track rigid bodies used to localize a robot to a patient. Isolating rigid bodies such as bone requires accurate segmentation and this study explores using diffuse laser reflectivity to accurately classify tissue. Lasers (Red, 650nm and IR, 850nm) intersected four material types; cartilage, ligament, muscle and metal surgical tools within a controlled cadaveric setup. Images were captured with an infrared CMOS sensor, pre-processed to isolate laser centres, and resized to test information requirements. Images for both laser types were scaled from 5x5 pixels to 30x30 pixels and trained on a convolutional neural network, GoogLeNet. At sizes above 15x15 pixels the IR laser had a higher classification accuracy reaching 97.8% at 30x30 pixels, whereas the red laser peaked at 94.1%. It was shown as not possible to qualitatively identify materials that were not trained in the network based on their probability outputs. Further work will be done to classify multiple points in a single scene as a step toward segmenting entire surgical views for markerless CAOS systems.