Generative AI and neural networks towards advanced robot cognition

Abstract

Enhancing autonomy and applicability of robotic systems across diverse scenarios, requires efficient environment perception. Conventional vision systems are highly performing but limited to simple tasks, while AI based ones require extensive data collection, processing and training. This paper presents a framework leveraging generative AI and Neural Networks to implement a dynamically updateable perception system. A multimodal conditional Generative Adversarial Network generates large image datasets which are automatically annotated by a Large Multimodal Model. A Convolutional Neural Network performs further dataset augmentation. A case study on the inspection of aircraft fuel tanks is used to showcase the potential of the approach.