The ProfessionAl Go annotation datasEt (PAGE)

We begin with the raw game data by accessing Go4Go²²2https://www.go4go.net, the largest Go game database. Game records in the database can be used for academic research with the owner’s permission. Some games were excluded from the raw records, including AI competitions, amateur games, handicap games, and games that ended abnormally. In the end, we obtained 98,525 games played by 2,007 professional players. We extracted metadata about each player’s birth date, gender, and affiliated association from three reliable online references: Go Ratings³³3https://www.goratings.org/en/, Sensei’s Library⁴⁴4https://senseis.xmp.net, and List of Go Players⁵⁵5https://db.u-go.net

Metadata related to matches and rounds were extracted from SGF files. We finally got 503 tournament categories (e.g., Chunlan Cup, LG Cup), 3131 tournaments (e.g., 1st Chunlan Cup, 15th LG Cup), and 342 rounds (e.g., round 2, final). We extracted data regarding matches and rounds from SGF files. Ultimately, we found various tournament categories (e.g., Chunlan Cup, LG Cup), tournaments, and rounds (e.g., round 2, final). Every tournament was manually labeled by a knowledgeable amateur player, including the category, level, and region. The tournament categories include championship, league, team, and friendly. The level of tournaments includes top-tier and regular. The region includes international and regular.

We used a Python script⁶⁶6https://github.com/pfmonville/whole_history_rating to calculate the player’s WHR ratings [41]. To obtain a fine-grained rating score, we took the following calculation approach. First, the initial number of iterations was set to 50, Second, by adding games day by day and performing one iteration, the rating was calculated for all time points. Notably, following the Go Ratings, the initial score was set to 3000. Finally, we obtained the WHR rating and the uncertainty. The uncertainty is higher if the player has just entered the leaderboard or has not played for too long.

The game records were analyzed using KataGo v1.9.1 with the TensorRT backend. In order to achieve a balance between accuracy and speed, the following strategy was applied: We performed 100 simulations for each move to obtain initial in-game statistics. After a move with a high fluctuation (greater than 10% win rate or 5 points), KataGo would reevaluate it with a simulation count of 1000. Finally, we obtained in-game statistics for all games, including win rate, score difference, uncertainty, ownership distribution, and recommended moves. We conducted our analysis on an NVIDIA RTX 2080Ti graphic card, which took about 40 days.

Fig. 2a presents the age distribution of the players in the different generations of the games. The average age of players before 1990 was between 30 and 50. As the decade progressed, the average age of players dropped rapidly to less than 30. The majority of competitions were completed by players in their 20s in the 21st century. It is evident from this trend that professional Go is becoming more competitive. Table I and Table II show the most frequent players and matchups. Most of the games were completed by these prominent legends, illustrating a long-tailed distribution of games played by different players.

Fig. 2b shows the number of games per year in the dataset. As we can see from the graph, it is growing exponentially over time. On the one hand, it indicates that the number of professional matches has increased in the last couple of years. On the other hand, PAGE does not appear to have recorded many games during the early years. Table V shows the tournaments with the greatest number of games played, mostly Japanese tournaments, except for the Chinese League A and the Korean League A. The main reason is that these tournaments have been held for a long time. As shown in Fig. 2c, resigned games have shorter lengths than non-resigned games. Fig. 2d to 2f show some trends of in-game statistics over time. We observe some interesting results from these schematics. With AlphaGo’s advent, professional players began to imitate the AI’s preferred moves, which is particularly evident in the opening phase (first 50 moves). One can observe the coincidence rate in Fig. 2d, which grew significantly after 2016. In addition, the non-opening phase is difficult to imitate, so the coincidence rate grows much slower than in the opening stage. According to Fig. 2e and Fig. 2f, the average loss of win rate and the average loss of score also decreased after AlphaGo emerged. Furthermore, players with higher WHR ratings have lower statistics than those with lower scores. These characteristics offer the potential to develop player performance analysis techniques.

Inspired by the work of Knapp [32] and Blanch [26], we conducted three experiments to explore the relationship between differences in participation rates and gender differences. First, we tested WHR scores using the negative hypergeometric distribution. Specifically, among $N$ female and $M$ male players, $R_{k}$ was the best female player ranked $k$-th among all combinations of participants. We calculated the 0.05% and 99.5% quartiles of the expectation distribution $r_{low}$ and $r_{high}$. If there was no gender effect in Go performance, the ranking $R_{k}$ of the $k$-th ranked female player should be between $r_{low}$ and $r_{high}$. Second, we made a binary observation for each sampling unit to determine the weight of the difference in rating scores that can be attributed to the difference in participation rates between male and female players, i.e., the explanation rate. Finally, we performed month-by-month calculations of the explanation rate to investigate changes in the gender difference over time.

In the first and second experiments, we selected the WHR ratings for December 2021 and excluded players with less than ten games. Finally, 1366 players were included in the calculation, with 1116 male and 250 female players. The experiments examined the ranking of the top 100 best female players, i.e., $k$ was set to 100. In the third experiment, we calculated the explanation rate month-by-month, where at least 50 female players entered the rating that month. To reduce the effect of the rating distribution on the results, we selected only the top 20% of female players each month and computed the average. For example, $k=10$ in April 1991 and $k=50$ in December 2021.

Fig. 3 shows the observed and expected rankings of the WHR scores for the top 100 female players. The observed rank scores (red line) lie outside the 99.5% quantile of the expected ranking (blue line). The actual score differences (red line) and the score differences attributed to different participation rates (blue line) are shown in Fig. 4. The unexplained differences between the actual and expected differences range from 221 to 346 points, with a mean of 272. In other words, only 23.2% to 56.1% (mean 44%) of the actual scoring difference is explained by the different participation rates of male and female players. Fig. 5 shows the month-by-month trend of the explanation rate of the actual rating differences. As can be seen, there is an overall decreasing trend in the explanation rate. It is worth noting that the maximum value of the explanation rate has been very high for several months. This is because Rui Naiwei reached second place in the world during this period, which is also the highest ranking ever for a female player.

From the three experiments mentioned above, we can find that gender differences do exist in professional Go tournaments, and participation rates barely explain the differences in WHR scores. Compared to chess, Go tournaments are a unique tool for studying East Asian cultures. However, there is little psychological research on the game of Go. We present the first analysis of the relationship between participation rates and gender differences through PAGE and expect to motivate more psychological work in the future.

This study explored the performance with several state-of-the-art deep learning models, including ConvNet, ZeroNet, KataNet, and Perceiver Transformer. Shao [45] designed CNNs to predict the moves of the board game, and it has achieved satisfactory performance in the RenjuNet database. This work uses regular CNNs, so we name it ConvNet. We call the network used in AlphaGo Zero as ZeroNet. Furthermore, the architecture used in KataGo is KataNet. There are no domain-specific improvements, such as ownership subhead or scoring distribution, included in our implementation of KataNet. Last but not least, the Perceiver [46] is a model built on the Transformer architecture. The model utilizes an asymmetric attention mechanism that extracts the input iteratively into a latent bottleneck, allowing it to scale to handle multimodal inputs. To ensure fairness, we set the CNNs to roughly the same depth and width, namely 6 successive blocks and 64 channels. In the Perceiver, we adjust the hyperparameters to maintain the same order of magnitude of parameter size as the CNNs.

To construct the blunder detection as a classification problem, we define a blunder move as an action with a 10% drop in win rate or a score loss of 5 points. Meanwhile, normal moves were consistent with the AI’s recommended moves. We extracted a sample of 982,323 blunder moves from PAGE. On the other hand, we randomly selected 1,138,135 samples from the normal moves to form the blunder prediction dataset. We divided the training, validation, and test sets according to the ratio of 7:1:2. All deep learning models were trained for ten epochs. The batch size was set to 128, the optimizer was Stochastic Gradient Descent (SGD), and the learning rate was 0.01. The experiments were conducted on an NVIDIA RTX 3060 GPU with 12G video memory. For each method, we report the accuracy and precision of each class in the test set.

Table IV reports the performance of the four methods in predicting blunders. Among them, three metrics indicate the overall accuracy, the precision of normal move classification, and the precision of blunder move classification. The performance of ConvNet is lower than the other three methods, probably because the shallower network architecture does not capture the information and features of the board state well. ZeroNet performs the best in terms of accuracy, while Perceiver Transformer has the best precision in blunder move classification. In our experiment, we only used the base features, i.e., the current state of the board. While multimodal analysis combining more in-game statistics has the potential to improve the performance of blunder prediction significantly.

First, we performed feature extraction and categorize all features into three groups: metadata features, contextual features, and in-game features. The detailed meanings of these features are illustrated in Table V.

XGBoost [47], Random Forest (RF) [48], LightGBM [49], and CatBoost [50] were selected as the training methods for the outcome prediction models. Three rating-based models were used for comparison: ELO, WHR, and TrueSkill [51]. In particular, we used the corresponding hyperparameters in the Python package by default and did not tune them. Although tuning these hyperparameters could improve the performance, our proposed approach has produced promising performance compared to other rating-based models.

Table VI shows the experimental results. We can see that WHR achieves an accuracy of 65.67% and an MSE of 0.2125 among all rating system-based outcome prediction models, which is the best performer. Compared to ELO, WHR performs slightly better because it takes advantage of the long-term dependence on game results. There is a lower performance in TrueSkill because it is more suited to calculating ratings for multiplayer sports.

The best performance of the machine learning methods using various features is CatBoost, reaching an accuracy of 75.30% and an MSE of 0.1623, which is much higher than WHR. The CatBoost model improves accuracy by 10% and MSE by 0.05 in almost every category. The improvement of the Others category is lower because it is relatively easier to predict, while the WHR method has an accuracy of over 72%.

We also conducted ablation experiments to check the validity of each modular feature using the CatBoost method. The results are shown in Table VII. The model using only meta-information features is 67.19%, which is slightly higher than the WHR method, suggesting that other attributes in meta-information, in addition to WHR ratings, play an important role in the prediction.

The accuracy was 70.99% when only contextual features were used. In comparison, the accuracy of 68.83% was achieved by using only in-game features, both higher than the state-of-the-art rating system-based prediction approach. By combining these features, the predictive ability of the model has improved. As demonstrated by the ablation experiments, the characteristics of each of our modules enhance the performance of the prediction system.

In traditional sports, there are many advanced in-game statistics, such as expected goals (xG) in soccer and total points added (TPA) in basketball. These advanced statistics enhance the fan experience and allow for better analysis of player performance. Even though we can better analyze player performance using our extracted in-game features, we can still do better. The prediction ability could be enhanced by adding more advanced in-game features. For example, time series analysis of win rate, score, and uncertainty, can be improved with well-developed techniques.

In Go matches, fans pay attention most to the styles and behaviors of different players. Taking Lee Changho as an example, he was able to defeat his opponent at the last minute through extreme steadiness and control. Alternatively, Gu Li tries to fight his opponents every chance he gets.

Modeling player styles and understanding human decision-making are challenging but fascinating tasks. Several potential applications can be found through its use, including targeted training, preparation, and AI-assisted cheating detection [52, 53]. This problem has only been discussed in a relatively small amount of board game literature. In this article, Omori [54] proposes that shogi moves could be classified based on game style and that AI can be trained to match certain game styles. McIlroy-Young [24] presents a personalized model for predicting an individual’s move and demonstrating how it captures human behavior at the individual level. The lack of proper analysis techniques and large datasets has contributed to the lack of noticeable success in style modeling and recognition. The advent of PAGE has helped to make progress on such issues.

In a result prediction model, a player’s winning percentage can only be predicted between two players. As opposed to this, a rating system can evaluate several players based on their relative strengths and is, therefore, more helpful in assessing a player’s strengths. There are currently only two types of rating systems used for board games: win-loss and time. It is possible for board games with more features to evaluate a player’s strength more effectively. Therefore, combining machine learning approaches with traditional rating systems has great potential.

Go matches are watched live on television or online platforms every year by millions of fans. A classic world Go tournament, on the other hand, has the tendency to last for over five hours in a room with just the host analyzing the game. This results in viewers getting bored with the show and stopping to watch. As AlphaZero has evolved, live Go broadcasts usually include the AI’s estimation of the position, which improves the viewer experience. While showing the win rate is essential to draw viewers’ attention, showing only the percentage of wins often leads to attacks on professional players because even top professionals’ games are rated very negatively by AI. Our proposed PAGE can change this situation. In live streaming Go matches, PAGE has the potential to develop technologies like automatic commentary text generation, real-time statistics, and result prediction that will enable a more detailed and nuanced analysis, significantly improving the viewer experience in live streaming.