Self-Supervised Production Anomaly Detection and Progress Prediction Based on High-Streaming Videos

摘要

Real-time production monitoring incorporating progress prediction and anomaly detection is essential for quality and efficiency. Traditional vision-based anomaly detection methods struggle to differentiate between production-related features and background noise, and fail to consider the heterogeneity of production stages. This paper introduces an integrated approach that merges progress prediction and anomaly detection, employing the Autoencoder Process Probability Embedding (APPE) method. APPE maps the distribution of images from normal production to a progress-related Gaussian Mixture Model (GMM), focusing on identifying production-relevant features while minimizing background interference through the proposed Spatial Activation Map (SAM). The proposed SAM improves the interpretability of the neural network by highlighting the specific features that influence the model’s decisions. The method is assessed through real-world datasets in the assembly of water valves and the production of commercial aircraft spoilers. The case study shows that our approach can achieve superior effectiveness compared to the benchmark, notably improving both task performances by integrating progress prediction with anomaly detection. Note to Practitioners—In many manufacturing settings, such as aircraft production, tasks that involve human-robot collaboration or high-precision manual assembly play a significant role. The ability to detect anomalies and monitor progress in real-time is critical for ensuring the quality and efficiency of production. The manual nature of these operations makes them challenging to monitor through in-situ embedded digital sensors, yet real-time operation videos are readily available. Vision-based production monitoring has been widely used in applications such as product surface inspection, but existing algorithms often face difficulties distinguishing between normal background variations and anomalies related to production. This paper introduces a new approach, called Autoencoder Process Probability Embedding (APPE), which integrates progress recognition and anomaly detection into a cohesive monitoring task, allowing the model to differentiate between background elements and features related to production. Although our method is demonstrated in production scenarios as case studies, the proposed SAM mechanism is versatile to be applied in other contexts with similar types of categorical labels.