SemanticAxis: Exploring Multi-attribute Data by Semantics Construction and Ranking Analysis

We refer to the technique of using vectors and their linear expressions to encode semantics and their transitions as semantic axis technique. Analogies in word embedding [28], such as man-woman, can be regarded as semantic axes. Heimerl et al. [14] helped understanding semantic differences between two corpora by checking word distribution in a cartesian coordinate system that is spanned by two semantic axes which is trained from these two corpora and describe the same concept. Liu et al. [23] expanded the descriptors of a defined semantics by detecting words that fall on each side of its corresponding word vector. Explainers [11] and InterAxis [19] learned linear functions from analysts’ continuous decisions to model the relationship between data attributes and abstract concepts in their minds. Our SemanticAxis follows a similar technical roadmap to the InterAxis. However, the tasks, hypotheses, and motivation of the two are completely different. To be specific, the InterAxis emphasizes on creating an interpretable reduced space by semantic modeling, but we focus on understanding the generated reduced spaces and its combination with data ranking.

For image or text, it is a simple and effective way to interpret their DR results by directly [35] or interactively [15, 20, 22] drawing the original data as annotation in reduced space. For other types of high-dimensional data, Ji et al. [17] applied parallel coordinates plots to identify hidden semantic features associated with recognized clusters. Cavallo et al. [5] explained the characteristics of a local region by ploting semantic curves of various dimensions centered on a certain data point. However, gererally, their approach requires out-of-simple support of the DR algorithm they used. Axisketcher [21] allowed analysts to draw a discretionary curve in reduced space and then helped them understanding the meaning of region that the curve passes through by expressing it as a combination of multi-segment linear functions. Stahnke et al. [34], Liu et al. [24], and we all uncovered the feature of local areas by analyzing the differences between the points inside and outside the areas. Compared to the former, we assign the differences a semantic explanation and apply it to the entire data set for rankings. Compared to the latter, we offer multiple interactions towards exploring the constructed semantics in depth. For example, we design a brush filter to support examing the distribution of semantics in reduced space with different granularity.

We highlight two kinds of interactions in the context of interacting with the DR model in reduced space: Parametric Interaction (PI) and Observation-Level Interaction (OLI)  [41]. PI refers to manipulating parameters directly in order to create a new projection. This presents a difficulty to novice or non-mathematically-inclined analysts. Typical examples include slider bars from Andromeda (PI view) [33], Star Coordinates [18], and SpinBox widgets from STREAMIT [1]. While OLI enables the analyst to directly manipulate the observations (data points), shielding the analysts from the complexity of the underlying mathematical models. Typical examples include StarSPIRE [3], Paulovich et al. [29], and Mamani et al. [26]. Recently, Jessica et al. [32] determined the differences, advantages, and drawbacks of PI and OLI, and drew the conclusion that these two serve different, but complementary. Semantic interaction is a similar concept to OLI whose interaction objects are also data points. It is just that the former puts more emphasis on the semantic interpretation of interaction. Semantic interaction follows the human-in-the loop pipeline  [8], in which analysts spatially interact with data models directly within the visual metaphor using expressive interactions, and then system interface provides visual feedback of the updated model and learned parameters also within the visual metaphor [6]. For example, Endert et al. [6, 7] enabled analysts to interactively generate a well-interpreted document space by semantic interactions, such as moving, highlighting, grouping, and annotating documents. These interactions are interpreted and mapped to the underlying parameters of a force-directed model [6] or a weighted MDS model [7]. In our SemanticAxis system, analysts can rebuild projection by directly modifying attribute weights, which is clearly a PI. The interaction of creating a semantic axis by selecting two groups of points belongs to neither PI nor OLI, since it does not interact with the DR model.

Gratzl et al. [12] summarized some common visual designs for multi-attribute rankings, including spreadsheet [10], point-based, line-based (e.g., parallel coordinates plot [16], slope graph [36, p.156], and bump chart [37, p.110]), and region-based (e.g., table with embedded bars [30], multi-bar chart, and stacked bar [9]) techniques. We adopted a line-based tenique: parallel coordinates plot, because it supports comparing the rankings of the same data point among various dimensions, multiple ranking criteria, and different time periods. Besides, dynamic weight adjustment is a widely used attribute refinement method [12, 40, 4, 38]. We implemented it in our system to assit analysts in customizing desired ranking criteria.

In this case, we aim at finding biased students according to their scores in an exam. The biased students refer to the students who go overboard on partial subjects but perform weakly in others. For example, some students may excel in natural sciences but do badly in humanities. It is necessary for teachers to find out these biased students, because they should be given guidance towards balanced development or be encouraged to dive into their specialties. Our data records the scores of 494 students at a middle school in Ningbo in a final exam involving nine subjects. The original scores are converted to z-scores to ignore differences in the data distribution between subjects. In this case, we pay attention to the performance differences between natural sciences and humanities. Traditionally, we consider natural sciences to include mathematics, physics, chemistry, biology, and humanities to include Chinese, English, history, politics, and geography.

We expect to construct a semantic axis to measure the performance differences. A reasonable design would be to set one end means “excel in the sciences” and the other represents “do well in humanities”. Since the target semantics is clear, we can construct the semantic axis in projection axis by dragging attribute rectangles directly. As shown in Figure 7, we set the value of the semantic axis on subjects in the two fields to a negative value and a positive value, respectively. Currently, the projection position of students on this axis should convey their degree of bias, that is, students at two ends may have a serious bias. To verify this, we select three representative students and check their rankings on each subject in ranking rows (Figure 7). We find that the No.14919 student ranked significantly better in natural sciences than humanities, the No.14740 student did the opposite, and the No.14760 student located in the middle of the semantic axis showed no obvious difference between the two. This indicates the correctness of the axis we constructed.

Next, we want to check whether the current reduced space captures the “biased” semantics. We select 60 students at each end with brush filter and examine their distributions in the reduced space (Figure 8). We find that these students are clearly clustered on the lower right side (\small{a}⃝) and the upper left side (\small{b}⃝), which suggests a positive answer to the previous question. Then we notice that the students prefer the sciences are distributed mainly in two distinct regions (\small{a}⃝). Hence, we build two semantic axes to understand their differences. As shown in \small{c}⃝ and \small{d}⃝, one region includs students who have a specific advantage in math alone, while the other region includ students who perform well in all subjects in the sciences, except for biology.

It is worth noting that there is no obvious cluster or other features (clues) in the original reduced space, which may cause analysts have no idea where to start the exploration. We overcome this by allowing analysts to build an initial impression of the reduced space by exploring the semantics in their minds first.

In this case, we explore the academic performance and rankings of the whole world institutions in computer science. Potential users include decision-makers who manage and plan the subject of computer science and students who look forward to choosing an ideal school. Our data comes from CSRankings [2] and is updated to April 2020. CSRankings devides computer science into 4 categories and 26 sub-areas. It scores 495 institutions on all areas based on their papers published in corresponding top conferences. These conferences are carefully chosen by senior domain experts. We set an individual linear scale for each area to range all scores between 0 and 100. Then we assess a synthesis score for institutions by weighted arithmetic mean on areas.

First, let us take a general perspective to check the distribution of institutions in reduced space. In the initial settings, the weight of all areas is 1/26, and the value of each attribute of initial semantic vector is equal. This means that currently, the position of points in projection axis and the size of the nodes in reduced space indicate the comprehensive strength of its corresponding institution. We discover a probable general pattern in the reduced space: the stronger the comprehensive strength of an institution, the higher its position. To verify this observation, we construct a narrow filter brush in the projection axis, and then move it from right to left by inches, while paying close attention to the movement of highlighted (brushed) points in the reduced space. We find, generally, the highlighted points move from up to down (Figure 9), which means that the vertical direction of the initial reduced space almost represents the comprehensive strength. Hence, the general pattern is approximately true.

Then we expect to understand the differences between different levels of institutions. As shown in the projection axis (Figure 9), we divide all institutions into three classes: first (top 80), second (81th $\sim$ top 50%), and third class (last half). Then we build two semantic axes with these three classes as endpoints (Figure 10). Unsurprisingly, institutions in upper class perform better than those in lower class in all areas. Nevertheless, the main differences between the first and second class lay in areas under System category, while the main differences between the second and third class lay in areas under AI category and Theory category.

We try to give a possible explanation for this finding. System category contains many old-line areas of computer science in which traditionally strong institutions have accumulated significant advantages. In contrast, most AI papers have been published over the past 10 years. The first class and second class almost stood on the same starting line. Besides, deep learning has swept through the whole academic, mobilizing the enthusiasm of almost all researchers in relevant fields, which makes the papers published in top conferences are no longer concentrated on a few top institutions (such as the first class institutions). The second-class institutions have participated in the competition extensively. However, the third class is still unable to catch the deep learning fast train, resulting in a widening gap in AI with the second class. As for areas in Theory category, their leading institutions are in the second class, leading to a comparable status between the first and second class while a relatively large gap between the second and third class.

Next we notice a cluster in the middle of the reduced space and intend to understand its meaning (Figure 11). We lasso this cluster and almost all the other points as the two ends of a SemanticAxis. In projection axis, we see that all the other areas are close to zero compared to the visualization area, which means that the feature of the cluster is “excellent in visualization area” and institutions with the same feature should be placed near the left. Further, we notice that there are four institutions in the lasso but away from the left, which means the current lasso are not accurate and these four institutions should not have been included in the cluster. Conversely, several institutions, such as Maryland-College Park (UMD), Stony Brook, Utah, and UC-Davis, are not in the lasso, but at the upper right corner of the reduced space. We infer that these institutions are excellent enough to follow the general pattern. This suggests another underlying semantics of the institutions in the focused cluster: they are mediocre in most other areas. All of these inferences are verified with the details provided by ranking rows. Now, we have shown that our system can help analysts to understand the precise semantics of clusters, correct inaccurate cluster boundaries, and detect and interpret outliers.

Finally, we expect to check how the rankings have changed in the field of AI. We set the weight of each of the five areas under AI category at 20% in weight editor and then divide the period from 2000 to 2020 into four segments at five-year intervals in weighted ranking row. We find that several Chinese institutions have made great progress (Figure 12). For example, Tsinghua University rose from 90th to 1st, Peking University rose from 110th to 3rd, and the Chinese Academy of Sciences rose from 56th to 4th. The top universities in the United States, such as CMU, Stanford, and UC-Berkeley, have always been among the best. The competition between China and the US in AI has become increasingly strong.