Seek for Success: A Visualization Approach for Understanding the Dynamics of Academic Careers

Careers have been widely studied in social science and data science. Most studies focus on the common patterns or similarity comparisons of career paths in terms of job-related attributes, such as title ranks and organizations. Traditional social science studies focus on a macro-level analysis, which uses cross-sectional analysis to learn inflow and outflows across occupations [47, 30]. Such point-to-point transition analysis fails to capture the careers from a long-term perspective to learn the impact of historical events on upcoming career performance. Works in data mining study from a micro-level to compare individual careers [72] and predict future jobs [70, 39]. It lacks a macro summary of careers, which is essential to understand the aggregated behavioral patterns in social science. Visualization studies use career data as a scenario in the event sequence analysis, including similarity analysis [32, 18, 20, 60] and visual sequence summarization [25, 24]. CV3 [20] is a system for comparing multiple resumes that assist recruiters in finding suitable candidates. Du et al. [18] and Jänicke et al. [32] developed two systems respectively to recommend similar career paths of students or musicians for a target individual. Instead of clustering the whole sequences, EventThread [25] and ET2 [24] use a fine-grained clustering to summarize career paths into latent stages.

The increased availability of academic profile data (e.g., Aminer [54] and Vispubdata [31]) has called for a renovated perspective to study academic careers. Existing works analyze these data from multiple perspectives such as publication, citation, and collaboration networks. Egoslider [65] and Egolines [78] distill the evolutionary collaboration networks to learn the academic interactions. Fung et al.designed a tree metaphor to visualize one’s publications [22]. VIS Author Profiles utilizes a template-based natural language generation to present a researcher’s profiles. Other works use novel designs (e.g., linked matrix and flower metaphor) to show the scientific influence of different research entities (e.g., researchers and publications) [50, 61].

However, most studies focus on career data, lacking the analysis of potential individual and social factors that can contribute to career success. A few visualization systems contain social factors (e.g., social networks) in career analysis. In the Interactive Chart of Biography [34], institutional and denominational ties are distilled to show the relationships of musicologists. ResumeVis [76] contains an ego-network-based graph that summarizes the co-working relations of the target individual. Nevertheless, these social networks have not been correlated with career success. In this work, we use multiple sources of academic profiles and propose a new visual analytics framework to understand how multi-factors affect academic career success from a dynamic perspective.

Event sequences are continually studied in the past decades in both social science and computer science with many shared research concerns. Since our focuses are analyzing the dynamic impacts of multiple factors on career success, we would discuss the most relevant works in both sequence mining and visualization.

Sequence mining in both disciplines can be summarized into three categories: pattern discovery [66], sequence inference [68], and sequence modeling [59], based on the survey of Guo et al. [26] and Piccarreta [45]. Sequence clustering is a universal technique to extract common sequential patterns using unsupervised learning. Xu et al. [71] summarized different clustering strategies into three categories: proximity-based [71, 46, 45], feature-based [27], and model-based [75, 41]. Specifically, proximity-based methods use the distance matrix to measure the similarity of sequences. Analysts first define and compute the pairwise dissimilarities between sequences and use clustering approaches to obtain sequential patterns. A critical step in proximity-based methods is to construct the distance matrix. Many sequence dissimilarity measures are proposed [53, 71] such as Euclidean distance and Levenshtein distance. Besides clustering on the entire sequences, several studies propose to use stage analysis to represent the sequence progression with more details [25, 24]. In addition to clustering, sequence inference (also noted as event history analysis in sociology) is another enduring topic. It estimates the influence of historical events or trajectories on upcoming events. Graphical models [10] and regression analysis [52] are widely used. We enhance the approach by Rossignon et al. [48] which combines sequence clustering with sequence inference to estimate the time-varying impacts of multiple factors on the upcoming career performance.

A variety of visualization techniques are also developed to analyze sequential data. Guo et al. [26] have proposed a comprehensive survey of event sequence visual analytics approaches. Here we mainly summarize the most relevant visualization techniques in our work. A large number of designs adopt an intuitive horizontal timeline encoding. Flow-based visualization is a typical representation with great scalability to summarize large-scale sequence data, including tree-based [42, 37] and Sankey-based [23, 63] structures. Two strategies are commonly used to reveal sequential patterns with fine granularity. One uses stage analysis to extract latent stages within the whole sequence to show progressions [25, 24]. The other directly visualize original individual sequences as horizontal lines [67, 13, 69, 16]. Recent work by Bartolomeo et al. [17] further uses layered directed acyclic network together with layout optimization to align specific events among sequences. Similar visual representations are adopted in storyline visualization [56, 55], where each line represents an actor, and the vertical position encodes groups of actors based on different events. Another set of studies use matrix- or list-based techniques to visualize event sequences that are scalable [79, 18, 64, 58, 62]. However, most of the state-of-the-art studies cannot be directly used in the study of academic career success. First, they focus on pattern extraction while the effects of multiple factors are of great importance in our scenario. Second, they consider the relative time which aligns sequences to the same starting point, while absolute time is also important to learn different generations of researchers. We have proposed novel visual designs to fulfill the above requirements.

The study of academic career success has been an enduring topic in multiple fields such as SciSci [21] and sequence analysis in social science [46]. Learning the time-varying impacts of multiple factors on academic career success can benefit the understanding of typical career patterns for social scientists and individual career development for general researchers. We have worked closely with a social scientist ($E_{A}$) to solve this research problem. He has been conducting multi-factor analysis with sequential data in different social domains. By working with $E_{A}$, we summarized the following concepts to characterize the problem of the multi-factor effect on academic career success.

• •

  Career Success, or Career Performance, refers to the outcomes (e.g., title ranks and incomes) of one’s working experiences [49, 8]. In academic careers, citations of a scholar’s research outputs are commonly used to measure career success [21].

• •

  Career Factors refer to potential factors (i.e., variates) that can affect one’s career success. It includes both individual (e.g., personal characteristics) and social factors (e.g., social relations) [74]. In academic careers, due to the data accessibility, we collected four factors that may affect career success based on previous empirical studies and our expert’s suggestions. Job title ranks, sectors (e.g., academia, industry, and government agencies), and research domains are individual factors. As collaboration is an essential type of occupation network in academia that can have significant effects on careers [21, 8], we regard collaborators’ individual factors as social factors [74].

• •

  Factor Categories are different groups identified based on the values within a factor according to user-specified definitions. For example, users may define people within the same value range of a factor as one category.

The analysis of dynamic multi-factor effects on academic career success is based on data from multiple sources, some of which require complicated and even manual data preparation. Considering the data availability and our familiarity, we focus on researchers from the visualization (i.e., VIS) community as a context to illustrate academic career analysis. We consider those who have published more than two TVCG papers in which the largest time gap is more than five years as potential VIS researchers after discussing with our expert. We then manually check and filter out those from other fields (about 90 researchers) and finally obtain over 1,100 VIS researchers. We use researchers’ names as inputs to search different data sources below, which are transformed into multiple sequences by each researcher for in-depth analysis.

• •

  Career data of researchers record the job-related attributes such as the institutions and titles. We collected it from LinkedIn [5], researchers’ personal websites, and their institutional webpages.

• •

  Bibliographic data is directly gathered from Aminer [54], which includes over 21,600 papers in total based on these researchers. It includes all the publication metadata of a researcher (e.g., authors, year, venue, title, and abstract) by year.

• •

  Citation data (by year) is crawled from Google Scholar [4] as a measure of career success [21].

Our goal is to analyze the dynamic effects of multiple factors on academic career success. Guided by the nested model of visualization design [43], we conducted literature reviews and frequent interviews with the expert to iteratively distill and refine the tasks. Finally, we form the analytical tasks into three levels [9] as follows.

The inter-factor-level task provides an overview of how different factors contribute to career success.

1. T1

   How does a factor influence career success over time? For longitudinal comparison, the expert wishes to know how the impact of a specific factor on career success develops over time. This could be related to the development of specific academic fields.

2. T2

   How do multiple factors differ in their impacts on career success? The expert wants to know the effects of different factors at a specific time as a cross-sectional comparison to determine the dominant factors influencing career success. Specifically, it is interesting to compare the impacts of individual and social factors.

The intra-factor-level tasks drive into one specific factor to compare the change of effects of different categories on career success over time.

1. T3

   How does a category within a factor change over time to affect career success? The impact of a category can change at different periods. It reflects the change of the roles of this category.

2. T4

   How do the categories within a factor differ from affecting career success? For each factor, different categories may have different sizes of impacts. Figuring out those with high impacts can help explain and provide guidelines for academic career development.

The individual-level task gives several concrete examples to help understand the factor effects on career success.

1. T5

   How does a researcher’s career path change over time? The expert wishes to identify different individuals from the data and study the multi-factor effects on career success from a micro perspective. Revealing the career changes of a researcher over time can help understand the different academic stages he goes over.

2. T6

   How do the different factors of a researcher change over time? Showing the evolution of career factors over time is essential to interpret the career development of a researcher and further validate existing rules or generate new hypotheses.

ACSeeker is a web-based application with three modules: a data preprocessing module, a data analysis module, and a data visualization module (Fig. 1). The data preprocessing module collects and cleans researchers’ career profile, publication, and citation data. Then it organizes these data into multiple sequences and stores them in the database. The data analysis module utilizes a novel framework to analyze the effect of multiple factors on career success. They form into the backend of the system and are implemented using Python and MongoDB. The data visualization module constructs a frontend application using Vue.js [7] and D3.js [12] with three views to support analysis.

After collecting multiple data sources (Section 2.2), we preprocessed the data in a semi-automatic way. For the career data, we organized each job into an event with a timestamp (by year) and an institution. We manually tagged the job titles and sectors in career data. Job titles were tagged into three ranks (i.e., junior, intermediate, and senior) based on researchers’ tenure in academic research (Fig. 2-B1). We also tagged three sectors: academia, industry, and government agency, based on the institutions (Fig. 2-B3). From the bibliographic data, we extracted the paper venues by year and classified them into twelve categories to represent different research domains based on [1] (Fig. 2-B2). We also extracted all the collaborators of a researcher by year to construct his ego-networks. For the citation data, we used Quartile [6] to divide each year’s citations into four ranks of equal size (Fig. 2-B1). In addition, we separated the top 3% citation researchers from the top 25% to inspect the pioneers in the visualization field.

Finally, multiple sequences are constructed for each researcher (Fig. 2-C). Career Sequences are composed of events defined by both title and citation ranks ordered by year. Domain Sequences consist of a list of paper venue categories with corresponding numbers of papers each year. Sector Sequences are sequences of sectors where researchers are affiliated over time. Citation Sequences are sequences of citation numbers by year. Meanwhile, we record all the collaborators of each researcher by year. We use their career, domain, and sector sequences as social factors. It allows us to quantify how a researcher’s collaborators can influence his career success. These sequences, along with the collaboration networks, are fed into our analytical framework.

A body of literature in social science has studied the effect of historical event trajectories on an upcoming target event [38]. Most of them reduce the historical sequences into summary indicators (e.g., event duration) [28], which fail to keep the full information in the original sequences. Sequence History Analysis (SHA) [48] is an innovative approach to preserve more complex sequential information in two steps. First, Sequence Analysis [46] is applied to identify representative patterns over the historical sequences. A distance matrix is constructed to document the pairwise distances between raw sequences. Using this matrix, the sequences are clustered into groups (i.e., categories) based on clustering algorithms (e.g., k-means). It retains the most representative sequential patterns over raw sequences and substantially reduces the computational demand in the subsequent multivariate analysis. Second, Event History Analysis [28] is used to analyze how these historical sequential patterns will affect the upcoming event. Different regression models will be applied to obtain the estimation of the effects.

However, the original SHA cannot be directly adopted in our scenario. First, it analyzes the impacts in a static manner by aligning sequences to the same starting point. However, the impacts of historical trajectories of factors could vary over time due to the development of the research field. Second, the SHA is supposed to analyze only one type of sequences, while in our scenario, multiple types of sequences are hypothesized to influence career success.

We have worked with our domain expert to enhance the SHA approach to support dynamic analysis of the impacts of multiple factors on academic careers over time. The whole framework consists of four components: sequence slicing, sequence clustering, multivariate linear regression, and cluster alignment (Fig. 2-D, E, F, G).

Sequence Slicing. We first improve SHA by slicing multiple sequences into different time windows. Given a long period, our expert hoped to inspect the time-varying impacts of different factors. Moreover, he stated the importance of considering the time-sensitivity of the impacts of historical sequences: the farther the historical event, the less relevant it is to the upcoming career performance. We thus apply the sliding window method [15, 73] to meet the two requirements. Each researcher’s career-related sequences (Fig. 2-D1) are arranged along the absolute timeline and sliced into different windows of a fixed size (i.e., size $w$, Fig. 2-D2). We aim to analyze the impacts of multiple factors in each time window separately. The window size and the moving step can be adjusted based on domain knowledge in ACSeeker.

Sequence Clustering. In each time window, we follow the Sequence Analysis method in the SHA approach to identify the most representative sequential patterns of each factor. It uses sequence clustering on each type of sequences (i.e., career, domain, and sector) respectively (Fig. 2-E). After comparing different clustering algorithms (e.g., k-means, k-medoids, agglomerative, DBSCAN, optics, and spectral), we finally chose k-means due to its optimal performance and efficiency. We use Euclidean distance to obtain the dissimilarity matrix, which is widely applied in social science with remarkable computational efficiency [53]. As our expert expected to adjust the number of clusters to obtain more meaningful results, the system allows users to customize the range of cluster numbers in k-means.

Multivariate Linear Regression. In each time window ($[t,t+w]$) (Fig. 2-F), we then conduct a multi-factor impact analysis on career success. We improve the SHA approach by applying ordinary-least-square (OLS) regression to incorporate multiple factors.

We have incorporated twelve independent variables (i.e., IVs, including individual and social factors) and a dependent variable (i.e., DV) based on our expert’s suggestions as follows:

• •

  $IV_{1}-IV_{6}$. Six categorical IVs are based on the clustering results of three types of historical sequences (i.e., Career, Domain, and Sector Sequences) of researchers ($IV_{1}-IV_{3}$) and their top collaborators ($IV_{4}-IV_{6}$) who have the most co-authored papers in a time window. We choose the top collaborator as a representative that potentially affects one’s career significantly (e.g., advisors or long-term collaborators). To include categorical variables as IVs in OLS regression, dummy coding [29, 3] is employed, transforming a categorical IV with $n$ categories into $n-1$ dummy variables [2]. Specifically, an arbitrary category of a categorical IV is chosen to serve as the reference category and all other categories are set to be the comparison or target categories. The obtained coefficient of each dummy variable means how a comparison category performs on the career success compared with the reference category. The obtained p-value shows the statistical significance of this comparison. As the choice of the reference category is arbitrary, we improve SHA by conducting a post-hoc analysis, which enables pairwise comparisons on the career success among all categories of a categorical IV as a panoramic view. Specifically, we traverse to set each of the $n$ categories as a reference category in turn in the regression. It results in a matrix-like table recording the coefficients and p-values of pairwise comparisons.

• •

  $IV_{7}-IV_{12}$. Besides the top collaborator, our expert suggested having a global summary of one’s overall collaborators in the model. We thus include two types of numerical variables. The first type ($IV_{7}-IV_{9}$) is the number of collaborators in each sequence cluster weighted by the collaboration strength. It is the product of the proportion of collaborators of this cluster and the normalized number of papers he collaborated over all the papers he published within the time window. The second type ($IV_{10}-IV_{12}$) counts the number of sequence clusters for each sequence type.

• •

  $DV$. We use the number of citations at the following year of the time window to measure career success [21]. We take the logarithm of the citations as DV since the raw citations each year are with a positively skewed distribution.

In each window, we compute the explanatory power (i.e., impact) of each IV (i.e., factor), which is measured by the difference of $R^{2}$ for whether adding this IV into the regression.

Cluster Alignment. In the last step, our expert wanted to align the same cluster in each sequence type across windows to learn its effect on career success temporally, which is not supported by SHA. We adopt a naive approach to label clusters by the event with the largest proportion in the cluster by year as it is the most representative one. We regard those with the same label as the same cluster and conduct the alignment across windows using these labels (Fig. 2-G).

In addition, for each researcher, we compute three diversity scores (i.e., career, domain, and sector types) of his collaborators by year using entropy ($n$ is the number of total event types):

The regression models of all windows (each includes p-values, coefficients and the difference of $R^{2}$ of each IV, and the predicted citations of each researcher) are fed into the ACSeeker for further analysis.

The Factor View (Fig. 3-A) is the primary visual component to show the regression results. It consists of two parts: (1) the Horizon Chart Group supports the inter-factor comparison on career success (T1, T2); (2) the Impact Timeline allows the intra-factor inspection within a factor (T3, T4). Users can have an overview of the time-varying effects of multiple factors and choose groups of interest for further study.

The Cluster View (Fig. 3-B) presents a list of categories (i.e., sequence clusters) chosen from the Impact Timeline. Three glyphs (Fig. 5-A, B, C) are designed to summarize the cluster of different sequences (T4).

Description: Each row shows a sequence cluster of a time window with a title listing the window, the cluster label, and the number of researchers within the cluster. Each glyph of the row summarizes the distribution of researchers within the cluster at one year. We have designed three glyphs to show the cluster summary of three types of sequences (Fig. 5-A, B, C). The career glyph uses a sunburst structure with two levels of hierarchy to show the career sequence event distribution. The inner ring shows three title ranks (i.e., junior, intermediate, and senior) with purple in three saturation categories (Fig. 3-C). The outer ring encodes five citation ranks in black at five saturation categories. The domain glyph is a doughnut chart that records the distribution of individuals’ most frequently published paper venue types in the cluster. We highlight the visualization venues as pink and others as grey to show visualization researchers’ domain diversity (Fig. 3-C). The sector glyph is also a doughnut chart showing the social sector (i.e., academia, industry, and government) distribution: green for the industry, blue for the academia, and yellow for the government agency (Fig. 3-C). The histogram on the right shows the citation distribution of the following year (i.e., DVs) of the sequence window. Users can compare the distribution of different clusters using the sequence list and choose a cluster of interest for further analysis.

Justification: Before designing three glyphs, we tried the proportional stacked area graph at first. For example, in Fig. 5-D, we used the same colors to represent three title ranks. We used a navigator on the left to show the citation ranks within each title rank. However, it was space-wasting and hard to be integrated with other career information. Thus, we used glyphs to summarize multiple information of a group which could be reused in Cluster View and Person View.

After choosing a cluster from the Cluster View, we use a CareerLine to visualize an individual’s careers and the effects of factors with folded and unfolded modes in the Person View (Fig. 3-C, T5, T6).

Description: In the folded mode (Fig. 6-A), the color of each circle shows the career title rank and the middle strip of the CareerLine represents the social sectors of the researcher, all with the same encoding in glyphs. Two flat gray areas distributed above and below the sector strip depict the collaborator and domain diversity scores of the researcher, respectively. The outer light blue area shows the predicted citations of the researcher based on the regression model. When hovering on a circle, a tooltip summarizing the researcher’s domain diversity is shown (Fig. 3-C4). The outer ring is a doughnut chart summarizing the paper distribution in different domains. The inner word cloud is generated based on the paper venue names from the bibliographic data. We break the venue names into separate words and count the frequency statistics of each word to reflect the research topics. The word size encodes the number of papers. Users can unfold the area of collaborator diversity score to study details about the researcher’s collaborators (Fig. 6-B). A multi-line graph shows the collaborators’ different diversity scores (i.e., career, domain, and sector). They can click one of the lines (i.e., diversities) to show the summary of the collaborators’ population distribution via corresponding glyphs (Fig. 5-A, B, C). We have provided sorting and filtering for users to choose researchers of interest. Users can rank them by average predicted citations or the collaborator diversity scores.

Justification: We tried to reveal all the collaborators’ career paths as curves around the target researcher, where the distance of the two career circles is proportional to the number of papers they co-authored each year. However, it caused severe visual clutters when the number of coauthors increased. In addition, according to our expert’s feedback, showing the raw career paths of collaborators was useless for comparing individuals and finding drivers that may affect the target researcher’s career success. Thus, we computed different diversity scores to summarize the information of collaborators at each timestamp.

We invited our expert in Section 2.1 to freely explore ACSeeker. We then summarize the observations and comments and form them into two cases to fully demonstrate the system.

We first interviewed our domain expert $E_{A}$. Although the initial purpose of the system is for social scientists to explore the dynamic multi-factor effect on academic career success, the dataset we use may also be of interest to visualization (i.e., VIS) researchers for their career developments. Thus, we also invited four VIS researchers ($P_{A}$-$P_{D}$) who had not used ACSeeker before. $P_{A}$ and $P_{D}$ are third-year PhD students with three-year VIS experience. $P_{C}$ is a researcher with six-year of expertise. $P_{B}$ just started her research as a first-year PhD student.

Procedure. Each interview for VIS researchers lasted about 60 minutes. We first introduced the background of the project (e.g., the research problem, the data, and the analytical tasks). Second, we used a comprehensive example to demonstrate visual encoding and interactions. Third, we introduced the cases in Section 5.1 to show the insights and the usefulness of the system. Fourth, they could freely explore the system in a think-aloud manner. Finally, we conducted a post-study interview to ask for suggestions on system workflow, analytical framework, and visualization and interactions. We recorded their comments and findings during the process. The feedback of the two groups of users is summarized into three categories.

System Workflow. Two groups of users all considered the system workflow clear. $E_{A}$ commented that ACSeeker followed and strengthened their traditional analytical workflow, “it provides a panoramic overview to compare the impacts of multiple factors, which makes the analytical findings more valid and straightforward.” He would follow the system logic to get familiar with ACSeeker from factor comparison to individual inspection. Then he would focus on analyzing the regression results (e.g., Factor View and Cluster View). The individual level was for case illustration and verification. For VIS researchers, those with statistical background (i.e., $P_{C}$) reacted similarly to $E_{A}$. Most insights he found were from the Factor View. However, others (i.e., $P_{A}$, $P_{B}$, and $P_{D}$) found it hard to understand the matrix encoding and mostly used the MatrixLine to choose clusters and explored from the individual level. They would conclude with the evolving patterns of careers of specific researchers. VIS researchers also had different analytical foci compared with $E_{A}$. They preferred to filter based on attributes (e.g., choosing collaborators from a specific domain) instead of clustering. They also wanted more information such as the real names of researchers and institutions to find role models. In general, there was a barrier for non-experts (i.e., VIS researchers) to explore ACSeeker due to diverse analytical foci and the lack of statistical background. It reflected that the target users of ACSeeker were still social scientists.

Dynamic Multi-factor Impact Analysis. $E_{A}$ appreciated the enhancement we added to the traditional SHA model, which made the whole framework more efficient. “The analytical approaches in social studies always follow a multi-step recipe. However, each step is troublesome that requires a lot of detailed processing semi-automatically. This automated framework indeed eases our burden.” He was particularly impressed by the sequence slicing and cluster alignment. It helped him analyze the impact from a dynamic perspective, which he had never tried before. He also suggested making the lengths of sliding windows more flexible to capture both long-term and short-term impacts.

Visualization and Interactions. All the users regarded the system as comprehensive to study the academic careers, which fulfilled all the analytical tasks. Most users liked the horizon charts, which gave a compact summary of multi-factors’ dynamic impacts. $E_{A}$ particularly liked the Impact Timeline, “each matrix is with the same form as the table of coefficients and p-values we obtained from the R program, but in a more intuitive visual representation. The MatrixLine and the animation to align clusters across time are creative.” For the CareerLine design, they could not remember all the design components at once. For example, at first, $E_{A}$ mistook the two gray areas for the same diversity score in a symmetrical manner, which actually represented two types of diversities. $P_{A}$ and $P_{D}$ could not fully understand the matrix encoding since they were unfamiliar with the regression model. Nevertheless, after our explanation and exploring for a while, they finally got the point. All the users commented that most of the interactions were intuitive and straightforward. $E_{A}$ appreciated the well-coordinated system with multiple views displayed, “I can brush and study the data across views conveniently.” $E_{A}$ and $P_{C}$ suggested improving the system readability, such as adding annotations for visual designs and abbreviations. $P_{B}$ wanted more interpretations related to the regression model.