A Human-Robot Interactive Mahjong Playing System Based on Visual Recognition Using a Convolutional Neural Network

Abstract

Mahjong is a popular tabletop game in China, Japan, and other Asian countries. As an incomplete information game, it is more complicated than other complete information games, such as Go, chess, and shogi. With the rapid development of service robot, there have been several human-robot interactive playing systems for complete information games so far, but mahjong is not the case. In this paper, a human-robot interactive mahjong playing system (HRMPS) was developed. HRMPS consists of five modules, including central host, robot players, visual kit, mahjong conveyor, and interactive software. The central host serves the communication between four robot players, which grab a tile on mahjong conveyor and recognize its face via visual kit, and make an action decision by themselves or by human opponents with the help of the interactive software. In HRMPS, to visually recognize a total of 27 different mahjong faces in uncontrolled conditions, a deep convolutional neural network was adopted to achieve an accuracy of 99.71% with a running time of 29ms. The experimental results tell that HRMPS is applicable in human-robot interactive mahjong game.