Understanding Players’ Interaction Patterns with Mobile Game App UI via Visualizations

User behavior analysis has been intensively studied in the game domain (Li et al., 2016; Kang et al., 2013; Lee et al., 2014). Li et al. analyzed players’ actions and game events to understand reasons behind snowballing and comeback in MOBA games (Li et al., 2016). These studies mainly focus on in-game players’ behavior analysis instead of their interaction with the game devices. The most similar studies come from web search (Blascheck et al., 2014; Kim et al., 2015; Huang et al., 2012), where statistical analyses of e.g., eye-tracking or mouse cursor data mainly provide quantitative results. Visualization techniques are developed to allow researchers to analyze different levels and aspects of data in an explorative manner, which can be categorized into three main classes, namely, point-based, area-of-interest-based, and approaches that combine both techniques (Blascheck et al., 2014). Among these methods, the aggregation of fixations over time or participants is known as a heat map that summarizes illustrations of the analyzed data and can be found in numerous publications. However, many methods proposed for desktop website analysis cannot be simply applied to mobile game apps since these methods are based on single-point interaction such as mouse or eye movements while mobile devices support intuitive direct-manipulation and multi-touch operations (Humayoun et al., 2017).

Timelines are frequently used to represent the interaction information. Previous solutions have provided static and limited representations of them (Carta et al., 2011). A new solution was designed to represent and manipulate timelines, with events represented by a level and a small colored circle (Burzacca and Paternò, 2013). It also includes vertical black lines among events that indicate a page change, which provides effective interactive dynamic representations of user sessions. Nebeling et al. presented W3Touch to collect user performance data for different mobile device characteristics and identify potential design problems in touch interaction (Nebeling et al., 2013). Guo et al. conducted an in-depth study on modeling interactions in a touch-enabled device to improve web search ranking (Guo et al., 2013). They evaluated various touch interactions on a smartphone as implicit relevance feedback and compared these interactions with the corresponding fine-grained interactions in a desktop computer with a mouse and keyboard as primary input devices. Bragdon et al. investigated the effect of situational impairments on touch-screen interaction and probed several design factors of touch-screen gestures under various levels of environmental demand on attention compared with the status-quo approach of soft buttons (Bragdon et al., 2011). To date, however, few empirical studies have been conducted on mining touch interactions with mobile game apps to understand players’ behaviors and further suggest mobile application UI designs via a visual analytics approach.

Many mobile applications, particularly game apps such as ARPG (Action Role Playing Game), introduce gesture operation to control the game character in the game app freely (Hesenius et al., 2014; Ruiz et al., 2011). However, no single gesture operation can resolve all the interaction issues in the mobile game application scenarios due to different screen sizes, individual behaviors, and the form of different controls used. Besides, gesture operation requires extensive learning. Thus, most existing mobile game applications, particularly role-playing games, adopt a virtual joystick and skill buttons for players’ interaction (Baldauf et al., 2015).

Previous studies that focus on the assessment of mobile UI design can be summarized into two categories, qualitative methods and quantitative methods. For example, when designing mobile app interfaces, targets are generally large to make it easy for users to tap (Anthony, 2012). The iPhone Human Interface Guidelines of Apple recommend a minimum target size of $44$ pixels wide * $44$ pixels tall (Clark, 2010). The Windows Phone UI Design and Interaction Guide of Microsoft suggests a touch target size of $34$ pixels with a minimum touch target size of $26$ pixels * $26$ pixels. The developer guidelines of Nokia suggest that the target size should not be smaller than $1$ cm * $1$ cm or $28$ pixels * $28$ pixels. Although these guidelines provide a general measurement for touch targets, they are inconsistent and do not consider the actual varying sizes of human fingers. In fact, the suggested sizes are significantly smaller than the average finger, which may lead to many interaction issues for mobile app users. Hrvoje et al. presented a set of five techniques, called dual finger selections, which leveraged recent developments in multitouch sensitive displays to help users select extremely small targets (Benko et al., 2006). The UED team of Taobao researched to determine hotspots and dead-ends and to identify the control size using the thumb¹¹1http://www.woshipm.com/pd/1609.html; they concluded that the minimum target size should be $11$ mm by single thumb operation to achieve an average accuracy higher than 95%. However, most existing works have focused on the vertical screen mode by single-hand operation, and only a few have discussed the landscape mode using both hands. In this work, we focus on the assessment of virtual joystick and skill buttons by leveraging players’ interaction data with the mobile game app to analyze the triggering and moving ranges of virtual joystick and skill set areas.

Regarding the quantitative methods, researchers have been working on modeling user perception and subjective feedback of user interface, e.g., the judgment of aesthetics (Miniukovich and De Angeli, 2015), visual diversity (Banovic et al., 2019), brand perception (Wu et al., 2019), and user engagement (Wu et al., 2020). Typically, a set of visual descriptors would be compiled to depict a UI page in terms of e.g., color, texture, and organization. Specifically, user perception data are collected at scale and corresponding models are constructed based on some hand-crafted features (Wu et al., 2019). However, feature engineering cannot ensure a comprehensive description of all the aspects of UI. Recently, deep learning has demonstrated its decent performance on learning representative features (Krizhevsky et al., 2012). For example, a convolutional neural network is adopted to predict the look and feel of certain graphic designs. Wu et al. leveraged deep learning models to predict user engagement level on animation of user interfaces (Wu et al., 2020). Similarly, perceived tappability of interfaces (Swearngin and Li, 2019) and user performance of menu selection (Li et al., 2018a) can also be predicted with the assistance of deep learning methods. However, the existing studies provide prediction scores of user perception towards different UI designs while in our work, we study how players experience in real-world practice via the current design of mobile game UI by visually analyzing their interaction patterns and shadow the design implication of mobile game UI interface.

Owing to the pervasiveness of multi-touch devices and wide usage of pen manipulation or finger gestures, a great number of researchers have conducted on stroke gestures analysis generated by users (Zhai et al., 2012; Tu et al., 2015; Vatavu and Ungurean, 2019; Leiva et al., 2018). Most of the related existing works focus on gesture recognition, which matches gestures with target gestures in the template library based on their similarity. For example, Wobbrock et al. developed an economical gesture recognizer called $1, which is easy to incorporate gestures into UI prototypes (Wobbrock et al., 2007). They employed a novel gesture recognition procedure, including re-sampling, rotating, scaling and matching without using libraries, toolkits or training. Yang Li developed a single-stroke and template-based gesture recognizer, which calculates a minimum angular distance to measure similarity between gestures (Li, 2010). Ouyang et al. presented a gesture short-cut method called Gesture Marks, which enables users to use gestures to access websites and applications without having to define them first (Ouyang and Li, 2012). In order to understand the articulation patterns of user gestures, Anthony et al. studied the consistency of gestures both between-users and within-users and some interesting patterns have been revealed (Anthony et al., 2013). Some works conduct research on different users, such as children (Anthony et al., 2012, 2015), elderly people (Lin et al., 2018), and people with motor impairments (Vatavu and Ungurean, 2019), aiming to improve user experiences on mobile device interactions.

In this work, instead of gesture recognition, we focus on analyzing user behaviors to find similar and preferred stroke gestures, i.e. similar operations generated by players when playing games. Considering different features of stroke gestures in terms of position, direction, shape and so on, we cluster stroke gestures to reveal users’ common behaviors by resampling a stroke gesture as a point path vector, followed by a definition of a distance function and clustering algorithms.

We have developed an application-independent Android program that can interact with the mobile OS. By detecting every touch event and retrieving the screen coordinates with the timestamp of the touchable screen, we can log all the touchable-screen actions through multiple functions, e.g., ACTION_DOWN (touch the screen), ACTION_MOVE (move on the screen), and ACTION_UP (leave the screen). We consider the players’ interaction data as high-level gestures, which are the trajectories of players’ fingers on the multi-touch screen that can be recorded as a series of points generated by the same finger of the player in one session of interaction. Particularly, when the player places a finger on the screen, a starting point $P_{0}$ is recorded. The player then moves this finger to other places, generating several corresponding points ($P_{1}$, $P_{2}$, …, $P_{n}$). When this interaction session terminates, the player raises his/her finger and ends the recording of the gesture. We define the gesture $g_{k}$ = [$P_{0}$, $P_{1}$, …, $P_{i}$, …, $P_{n}$], where point $P_{i}$ is described in terms of ($x_{i}$, $y_{i}$) with corresponding timestamp $T_{i}$ indicating the corresponding time-lapse from the beginning. Each gesture trajectory has an associated id, which records the current session the program identifies during the interactions.

The basic design principle behind our approach is leveraging or augmenting familiar visual metaphors to enable game experts to focus on analysis (Li et al., 2018b). Considering the gesture data being analyzed and the above-mentioned requirements, we visually encode the data in a manner that ensures that the patterns and outliers are easily distinctive without overwhelming the analysts.

We define a gesture as a series of finger actions, typically starting with a “finger down”, followed by several “finger moving”, and ending up with “finger lifting”. We adopt a timeline-like metaphor to align all the actions of the fingers in a radial layout, presenting the overview of the distribution of finger actions intuitively, as shown in Figure 3. We choose a radial layout because it allows users to analyze high-level interactions (e.g., periodic behaviors) and compare the gesture patterns in different stages in a more concentrated manner. Meanwhile, the mobile game interface could be placed inside the radial circles, helping analysts better link temporal events with the corresponding interaction dots and spatial trajectories on the screen. We apply three concentric circles to denote the three fundamental actions of fingers, namely, “finger touching” in the most inner circle, “finger moving” in the second inner circle, and “finger lifting” in the outermost circle (R.1). The spatial position of each finger action is determined by its timestamp and the entire recorded period is considered in a clockwise direction. One complete and continuous gesture is represented by a Bezier curve that links the starting action (finger touching) and the ending action (finger lifting) with a series of finger movements distributed between the two actions. We also encode gesture durations into the parameters of two controlling points of the Bezier curve, i.e., the height of the curve corresponds to the related gesture duration. We also provide heat map visualization as a qualitative complementary to the proposed gesture visualization.

In addition, to encode the gestures to concentric circles, we filter out gesture trajectories based on spatial locations, e.g., we can visualize the trajectories that correspond to a particular area of the mobile app interface (R.1). Particularly, we design a spatial-based query to link the trajectories to the corresponding finger actions along the concentric circles (see the selected area). We directly integrate the UI design into the visualization. On the left side of the view, the interaction within the area controls the orientation and motion of the game character in the app, and the right side, i.e., the five rounded areas correspond to different skills of the game character. Figure 3(b) presents the use of the spatial-based query for a relatively long duration of operation, which corresponds to a gesture trajectory. Most of the finger movements occur on the left side, whereas quick taps happen on the right side.

Since the skill button interaction represents certain semantic information, i.e., different skill button indicates different operations on the game character, we can define the high-level interactions in the skill semantic space. As shown in Figure 3(c), analysts can self-define a region, e.g., a circle to surround an area and assign certain semantic information to the area, and the corresponding finger actions within the area would be automatically mapped onto the semantic axes on the outermost semantic circles, which supports the interactive exploration from the skill semantic perspective (R.2).

Conventionally, to identify the typical interaction patterns on the mobile game app screen, the heat map is intensively used. One obvious advantage is the lack of quantitative feedback by using a heat map that only conveys a sense of qualitative density information. After discussion with the game experts, we propose an interactive clustering method based on the interaction data (R.3). Particularly, we allow analysts to select the original interaction dots with a certain radius $R_{0}$ and the system automatically calculates the clustering center $P_{0}$ of the selected interaction dots and choose the area with different confidence coefficients, i.e., identify the most appropriate center and size of the area that meets certain confidence coefficients. In this way, a new clustering center $P_{1}$ could be generated. We further sort the interaction dots based on the distance between $P_{1}$ and the interaction dot $d_{k}$ and the longest distance is considered as the new radius $R_{1}$ (Figure 4).

Once we determine the region for a certain confidence coefficient, we then extract the gestures within the region to understand the underlying semantics of the gestures and reveal the common interaction behaviors of players. Note that different gestures contain various number of interaction dots, we need to resample gestures to ensure that all the gesture trajectories are directly comparable. Given a gesture $g_{k}$, we assume its original number of interaction dots is $M$ and the objective is to sample $N$ interaction dots from $g_{k}$ with evenly spaced sampling. Specifically, the original gesture is represented as $g_{k}=[P_{0},...,P_{i},...,P_{M-1}]$ and $P_{i}=(x_{i},y_{i})$ and after sampling, we get a new gesture $v_{k}=[P_{0}^{s},P_{1}^{s},...,P_{i}^{s},...,P_{N-1}^{s}]$, where $P_{0}=P_{0}^{s}$ and $P_{M-1}=P_{N-1}^{s}$. They are subjective to

In other words, we add the first interaction dot $P_{0}$, and then add a new dot sequentially until $v_{k}$ covers $N-1$ dots, followed by adding the last interaction dot. Those new interaction dots is generated from the original gesture $g_{k}$ by using linear interpolation. The next step is to measure the distance between two gestures. Particularly, given two gesture sampling vectors: $v_{k}(P_{i}=(x_{ki},y_{ki}))$, $v_{h}(P_{i}=(x_{hi},y_{hi}))$, and considering the absolute distance and directions, we combine two distance functions, i.e., Euclid Distance and Cosine Distance with adjustable weights, and use K-Means to all sampling gesture vectors. The distance functions are defined as Euclid Distance:

and Cosine Distance

where

and

The education backgrounds of the above-mentioned participants range from computer science, electronic engineering to art designs. We collect information about the participants’ mobile phone usage, including their mobile phone operating systems, the size of their mobile screens, and the frequency of playing mobile games per week. All of them have the experience of using both IOS and Android mobile operating systems. The size of the mobile phone screens they use ranges from $3.1$ to $3.5$ inch to above $5.0$ inch. Regarding mobile game experiences, most of the participants (80%) play games for fun, usually $3$ – $4$ days per week. Three of the participants consider themselves experts in playing mobile games, five have intermediate-level gaming experience, and the others are novices. Their gaming expertise is based on the number of mobile game apps they have ever played similar to our testing mobile game app and all of them have no prior knowledge of our mobile game app, i.e., nor have they seen it or played with it.

The mobile game app we choose for our study is a type of ARPG, where the player controls the actions of the main game character immersed in a well-defined virtual world and resists attacks from in-game monsters. The orientation and movement of the main game character are controlled by the player through a virtual joystick placed on the left side of the mobile screen, while the skill release controls are placed on the right side of the screen, usually consisting of five skills (i.e., one Normal Attack in the right-button corner surrounded by the other four skills). We have studied different mobile applications (e.g., “address book”, “2048”, “Angry Birds”, “Temple Run”) and identified that they all share the same set of basic down/move/up actions. However, “Angry Birds” only involves limited events of finger move (e.g., launch a bird) and finger down (e.g., make birds explode). Therefore, we choose the virtual joystick mobile game that involves lots of finger down/move/up events fully engaging players via interactions with both hands and has a proper duration ($2$ – $5$ min on average) and different levels of difficulty. Thus, the resulting gesture interactions are diverse enough for our experimental analysis.

The objective of the first case study is to differentiate novice players from the expert ones based on their interaction data with the mobile multi-touch screen when playing our testing mobile game app. For the first case study, we recruited two novices (one female student, age: $25$ and one male student, age: $23$) and two expert gamers (male students, age: $20$ and $25$, respectively) to compare their interaction skills by using our approach. A single gaming session was conducted with each participant for $5$ minutes. They were firstly given a brief overview of the basic operating rules of the testing mobile game app. Then, each participant played for three consecutive rounds, and we only collected the operation logs of the last two attempts. The first attempt served as a training session to help the participants familiarize with the testing mobile game app.

As shown in Figure 5(1), we applied our visualization approach to the interaction data of a novice player and identified that the player has only three finger movement trajectories, indicated by (a), (b), and (c), respectively. Among the three movement trajectories, (b) and (c) occur within the first $10$ seconds, followed by a long-lasting movement of over $107$ seconds that continues until the end of the game session. We then observed how the player interacted with the skill buttons by the five corresponding skill concentric semantic circles. Most of the five skill buttons are triggered simultaneously in clockwise order with the Normal Attack (the biggest button) triggered first followed by the other four skills, as indicated by the green rectangles. Since the four skills have several seconds for cooldown, they cannot be triggered immediately if they have been triggered and the Normal Attack is frequently used. We can conclude that the operation of the novice player involves a continuous movement on the left side of the screen to control the orientation and movement of the game character in the mobile game app and consecutive release of the five skills in a clockwise direction. The left (orientation and movement) and the right operation (skills) are not combined well. In other words, the player did not have a good strategy to combine the orientation control and the movement of the game character effectively with the other five skills to defend the game character against attacks.

For comparison, we visualize another expert player’s interaction data, as shown in Figure 5(2). Following the same approach, we first focused on the long-last movement trajectories, which are defined by relatively long and high curves that connect the corresponding “starting” dots (finger down) and “ending” points (finger up). We identified that at the timestamp of each long-lasting movement, the skill set on the right side of the screen is triggered in a regular pattern, i.e., the Normal Attack is typically accompanied by the other four skills that require a certain cooldown time. Thus, the efficient combination of the long-lasting movement to control the orientation of the game character and the release of the skill set results in the player winning the game.

After comparison, the game experts maintained that the four skills, except for the Normal Attack, are lower in operations because of that “a good operation focuses more on joystick control and movement.” (E.1) By contrast, a weak operation depends significantly on the Normal Attack compared with the other skills, and involves fewer finger movement on the left side of the mobile screen. From the visualizations, the game experts commented that “the weak operation continuously utilizes the Normal Attack during the finger moving and touching operations” (see the most inner semantic circle in Figure 5(1)) “while the good operation applies the Normal Attack regularly and intermittently”.

In this case, we demonstrate how the game experts leverage our visualization approach to track the potential improvement of an individual player’s skills when he/she takes a series of attempts of playing a mobile game. The game experts invited another two novices players (a male student with the age of $28$ and a female student with the age of $24$) and asked them to carry out a series of attempts consecutively. Particularly, the participants were first introduced to get familiar with the basic operations and then played the mobile game several times. After finishing the attempts, their interaction data were recorded each time and the game experts conducted an interview on each of them, separately taking a note of each interviewee’s response.

Figure 6 shows a case where a participant (male, age: 28) played the game four times, of which the first two attempts failed to accomplish the game task. To be specific, in the first two attempts, although the participant applied all the skills, they were conducted successively. We observed that continuous touches occur on the second, third, and fourth skill buttons, which should be avoided since that except for the Normal Attack, all the other four skills have a cooldown time that disables the corresponding skill for a certain duration. Another phenomenon that can be witnessed is that nearly all the skill triggering happens simultaneously and there is no correlation between the movement control by the joystick and the skill operations. On the other hand, the last two results correspond to successful attempts. The combination of movement control and skill release becomes more obvious. The triggering time intervals between different skills are elongated, as shown in Figure 6 (d). Furthermore, the movement trajectories are denser and more concentrated in Figure 6 (d) than that in the previous attempts. “In the beginning, I was quite nervous and did not know how to handle so many monsters so I just click on every skill buttons,” said the participant, “I soon realized that without any strategy I would never win the game”, so he began to coordinate his both hands.

Another participant made three attempts. In the first two attempts, the participant also failed to finish the gaming task. Particularly, in the first attempt, the participant only used Normal Attack and the other two skills. Although the movement was quite intensive, there is no combination with the application of skill release. In the second attempt, the participant began to use more skills, but in a random way. Taking a close look at the triggering timestamp of the skills and movement control, the game experts cannot identify any correlation. In the last attempt, the participant commented that “I feel like knowing how to play” when he began to pay attention to the combination of the joystick with the application of different skills. For example, when there was a relatively long-lasting movement, some skills were released immediately. As shown in Figure 7, the ratio of Normal Attack significantly drops, and more operations are given to the joystick and are distributed in the four skills evenly.

This case focuses on identifying the common behaviors of players’ interaction with the mobile game app and further verifying the current UI design of the mobile app interface in real usage scenarios. To approach this, the game experts covered the main operation regions of the mobile game app interface to ensure that the participants operate based on their own habits for an objective evaluation. As shown in Figure 1(b), the size of the covered region is approximately $3$ cm * $3.5$ cm. The game experts invited all the participants and asked them to use only two thumbs to operation on the game avatar in the mobile game app for five rounds, no matter success or failure for accomplishing the gaming task. During the gaming process, all the participants were asked to keep their thumbs in a natural curve and use their finger bellies to touch the screen while maintaining the stability of the mobile devices. The mobile device used has a width of $11.07$ cm and a height of $6.23$ cm. The resolution of the device was $1920$ * $1080$ pixels and the OS is Android. In total, we recorded $45$ independent interaction data, in which about $400$ gesture trajectories were considered valid. After the experiments, the game experts conducted an interview with the participants to collect their subjective feedback when playing the mobile game app.

Virtual Joystick UI Verification. The game experts first focused on the triggering area and moving region of the joystick on the left side of the mobile screen. We applied interactive clustering on all the touching sampling dots generated by the participants’ interactions. To obtain the minimum coverage of the triggering area of the virtual joystick, we covered the interaction area with a yellow circle with the radius of $R_{0}$. Through the interactive clustering, a new radius $R_{min}$ is determined. To obtain the upper boundary of the triggering area, we widened the initial coverage and adjusted the confidence coefficient to 99%. Another new radius $R_{max}$ was observed. Meanwhile, the distance to the left boundary and the bottom boundary is recorded to locate the specific position of the new center. The experimental results indicate that the lower and upper boundary of the triggering area is $15.61$ mm and $25.42$ mm based on our method and the distances to the left and the bottom boundary of the screen were $15.86$ mm and $13.85$ mm with the confidence coefficient of $95\%$. The same procedure can be applied to determine the moving range of the joystick. We summarize our findings in Table 2 and Figure 8.

To further identify the major trajectories using the virtual joystick, we clustered the gestures that lie within the boundary of the moving area of the virtual joystick based on the proposed clustering method. Eexperimental results show the patterns that occupy the majority of the trajectories. To be specific, apart from those gestures with short distances, the gestures with relatively longer distances are clustered into $13$ clusters, representing the main gestures that demand high workload, i.e., moving fingers with a relatively long distance. As shown in Figure 9, the new boundary of moving area of the virtual joystick has covered the most frequent gestures that take a relatively long distance for fingers to move, which indicates that the above experimental scale suggestions can support a free movement according to the participants’ operation habits.

Skill Set UI Verification. Following a similar procedure, the game experts determined the scale and position of the skill responding area by adopting interactive clustering that covers all the touching sampling dots in the skill set area. Table 3 gives a summary result of the experimental results for the skill set. Particularly, with the confidence coefficient range of $90\%$ and $95\%$, the spacing among skill buttons is in the range of $7.6$ mm – $10.25$ mm. Due to the fact that players may easily confuse with the middle two skill buttons, the spacing should be relatively larger between c and d. Similarly, the distances between the centers of the skill buttons and the right/bottom boundary of the mobile screen are in the range of $6.09$ mm – $24.10$ mm and $5.93$ mm – $22.85$ mm, while the distances between the center of the Normal Attack and the right/bottom boundary are in the range of $9.66$ mm – $9.74$ mm and $9.36$ mm – $9.44$ mm, as indicated in Figure 10.

We first asked the game experts to evaluate the insights identified by our visualization approach. E.1 – 2 reported that “the visualization displays the interaction patterns of players intuitively”. Conventionally, the game team needed to watch video replays, and manually marked segments of interactions: “the time we spent on the entire process was about $30$ minutes since sometimes we had to replay a certain video segment many times,” said E.1. Our method can visualize the spatiotemporal attributes of the gesture interactions, making it easy for the experts to interact with players’ behaviors with the mobile game app instead of going through the entire video replay, which largely shortens the analysis time (now around $5$ minutes for each session). The UI design experts were very satisfied with our approach’s ability to spot potential UI design issues for the joystick and skill areas, which serves as a complementary to traditional methods such as survey other game competitors, “it helps me obtain detailed UI suggestions by interacting the interface,” said E.4, “I am now more confident to draw my UI design conclusions since my subjective feelings can be confirmed,” said E.5.

We also discussed with the five game experts which component(s) of our approach can be directly applied to other kinds of games or apps and which one(s) need customization. They all appreciated that our approach has been already applicable to any kind of ARPG apps since their functions are quite unified, mainly around joysticks and skills. The only place that needs customization is the middle part of the interface, i.e., the mobile game app interface that provides gaming contexts.

Our visualizations have provided some design implications for the UI design of all kinds of ARPG apps. Players spend most of the time operating with the joystick, which is also the most energy-consuming; therefore, it has the highest requirements for flexibility and fineness. “The purpose of the virtual joystick is to control the distance and direction of movement and the orientation of the game avatar,” said E.4. Participants also reported their requirements for the virtual joysticks include: 1) flexible and fast control of the position of the game avatar; 2) precise control of the distance and direction of movement; 3) lasting operation does not consume too much energy. The results from our visualizations also indicate that the finger movement areas should be sufficient enough to ensure that most areas of the mobile screen are responsive, “it is best to move anywhere you want,” said one participant (P1, male, age: 25).

In terms of the skill interaction design, UI design experts reported that their design principles are to support clicking on the graphical representations of the buttons to release skills and to display cooldowns when the skills are not available. From the perspective of participants, they wanted the design of the skill buttons to meet the following requirements: 1) to release skills quickly and easily; 2) to accurately release skills with less attention; 3) to quickly know when skills will be available. Through the quantitative experimental analysis of participants’ interaction data, UI design experts found that participants are easy to locate high-frequency buttons through the edge of the screen, which should be located along the “fan-shaped” curve. Other lower-frequency operation buttons such as switching roles can be placed in the area near the $41.3$ mm arc, facilitating easy clicking and locating. In addition, the participants reported that the middle two skill buttons are more easily noticed during operation and are suitable for major/important skills, while skills near the border on both sides are relatively more suitable for auxiliary skills such as dodging.

Our game experts also pointed out several directions for improving our visualization approach. E.1 and E.2 hoped that we could take in the in-game video and metrics from the mobile game app as a whole, enabling a comprehensive analysis. It is for ensuring the high “consistency of analysis conclusion.” Meanwhile, the game experts commented that “the method has the potential to be developed into a training tool to support a retrospective analysis of players’ performance.” The UI design experts (E.4 – 5) also hoped that we could include more in-game contents. For example, a proper number of monsters and stable in-game fighting duration together with appropriate UI design can not only prevent the players from “being in a state of high operating frequency all the time”, but also “bring the players a sense of satisfaction”.