Continuous Perception for Classifying Shapes and Weights of Garments for Robotic Vision Applications

摘要

We present an approach to continuous perception for robotic laundry tasks. Our assumption is that the visual
\nprediction of a garment’s shapes and weights is possible via a neural network that learns the dynamic changes
\nof garments from video sequences. Continuous perception is leveraged during training by inputting consecutive frames, of which the network learns how a garment deforms. To evaluate our hypothesis, we captured
\na dataset of 40K RGB and depth video sequences while a garment is being manipulated. We also conducted
\nablation studies to understand whether the neural network learns the physical properties of garments. Our
\nfindings suggest that a modified AlexNet-LSTM architecture has the best classification performance for the
\ngarment’s shapes and discretised weights. To further provide evidence for continuous perception, we evaluated
\nour network on unseen video sequences and computed the ’Moving Average’ over a sequence of predictions.
\nWe found that our network has a classification accuracy of 48% and 60% for shapes and weights of garments,
\nrespectively.