Computer and Vision Aided Mapping in Farmer's Market and Visually Aided Shopping Strategy for Visually Impaired Individuals

Abstract

Existing guide equipment and methods are not fully suitable for the visually impaired (VI) shop in the crowded and narrow farmers' market. Aim assist VI to purchase food more conveniently, this paper proposes computer and vision aided mapping (CVAM) algorithm. It utilizes threshold function to quickly convert the CAD layout of the farmer's market into a navigation-specific grid map. Furthermore, CVAM utilizes the A-star method to traverse each stall, and employs a classification camera based deep learning to identify and record the products on stall. Additionally, this paper proposes a visually aided shopping (VAS) algorithm that allows robot minor seller to select and weigh products. It employs the KCF method to track the seller's hand movements and uses a classification camera to ensure that the products selected by the seller are consistent with the VI's expectations. Subsequently, it uses BP neural network to read the electronic scale indication and feed it back to VI. The experiments conducted in this study confirmed that the guide robot designed in this study is not only capable of navigating the farmer's market for VI, but also greatly improves mapping efficiency compared with general mapping method. Particularly, VAS successfully supervises the entire process of seller selecting and weighing products, preventing them from deliberately deceiving VI.