Towards automatic robot-assisted microscopy: An uncalibrated approach for robotic vision-guided micromanipulation

Abstract

Micromanipulation during live microscopic imaging relies heavily on good manual controls, dexterity, and hand-eye coordination. However, unassisted manual operations in these procedures greatly limit the speed, repeatability, and ease of operation. This is especially challenging in the case of microinjection where the insertion path needs to be in precise alignment with the imaging plane to avoid damage to cells. In this paper, we proposed an assistive robotic system that facilitates micromanipulation under microscopy. This comes in the form of intelligent robotic vision and guided manipulation. Using user-selected patch similarity, the system registers target templates and provides online coordinated depth compensation that ensures in-plane microinjection without the need for any prior calibration. This vision-based auto-registration approach readily integrates to any existing microscope system uncalibrated. It can also work as a standalone imaging solution with any general digital microscope camera. Experiments show that the similarity-score based depth compensation performed better than the uncompensated method. The method was shown to self-recover from an unfocused position. By robotizing conventional microscopy and micromanipulation procedures, we hope to address traditional latent needs and open up new possibilities in the ways experimental biology is performed.