Adaptive Multimodal Learning for Robot Decision-Making in Dynamic Environments

Abstract

This paper introduces an innovative adaptive multimodal decision-making paradigm for uncertain dynamic environments based on attentional fusion and Deep Reinforcement Learning (DRL). The model increases decision accuracy and reliability, especially in noisy or partially observable conditions, using visual, audio, and haptic inputs. The attention model continuously evaluates each modality's importance, which allows real-time task-specific adaptability. Simulation experiments confirm the effectiveness of the framework, resulting in better classification accuracy and smaller latency than in baseline models. The results testify to the usability of the given system for real-time high-performance robotic applications of interpretable results.