Deep video anomaly detection in automated laboratory setting

Abstract

Laboratory automation integrates robotics, machine learning, and computer vision to enhance precision and efficiency while reducing costs. Despite its pivotal importance in fully automated experimentation, automatic monitoring of procedures is an overlooked task. In this paper, we aim to address this shortcoming by developing a learning method for detecting anomalies in the laboratory setting. To formalize the problem, we focus on the liquid transfer task as a major task in laboratory automation, given the frequent need for reagent mixing and solution preparation. We introduce a novel video anomaly detection framework that leverages the robust CLIP features in conjunction with a transformer encoder. Through an array of experiments and ablation studies, we demonstrate that our proposed method surpasses current state-of-the-art anomaly detection techniques adapted to laboratory automation by a notable margin of over 11%, achieving an impressive AUC of 98.79% for video-level anomaly detection. • Introduced a deep video anomaly detection framework in automated laboratory setting. • The model estimates the score for the video and individual pipettes. • The model exhibits notable performance when tested against the evaluation samples.