AMAF 휴리스틱을 적용한 삼목게임

All-Moves-As-First Heuristic applied to the game of Tic-Tac-Toe

Monte-Carlo Tree Search (MCTS) is a best-first search algorithm to evaluate states of the game tree in game playing, and has been successfully applied to various games, especially to the game of Go. Upper Confidence Bounds for Trees (UCT), which is a variant of MCTS, uses the UCB1 formula as selection policy, and balances exploitation and exploration of the states. Rapid Action-Value Estimation (RAVE), which is a All-Moves-As-First (AMAF) heuristic, treats all moves in a simulation as the first move, and therefore updates the statistics of all children of the root node. In this paper, we evaluate the performance of RAVE and UCT playing against each other in the game of Tic-Tac-Toe. The experimental results show that the first player RAVE is much inferior to the second player UCT (13.0±0.7%), on the other hand, the first player UCT is far superior to RAVE (99.9±0.1%).