Understanding the Advisor-advisee Relationship via Scholarly Data Analysis

Our large-scale dataset of advisor-advisee relationships is generated by applying a deep learning technique, stacked autoencoder, on the DBLP dataset (Wang et al., 2016, 2017). The specific steps for the advisor-advisee dataset acquisition are as follows:

1. 1.

   Construct the collaboration ego-network for all scientists in the DBLP dataset.

2. 2.

   Clean the network: Following Sinatra’s (Sinatra et al., 2016) steps, we only choose the scholars who have published at least 5 papers and who have a publication career span of at least 10 years in the DBLP dataset.

3. 3.

   Extract the required features for training as unlabeled input of the model: Compute the personal properties (i.e., academic age, number of publications) and collaboration properties (i.e., collaboration times, collaboration duration, times of first-two authors, and cohesion of collaboration) for each scholar in the first 8 years of collaboration, then use these normalized features as the input of the model. The Back Propagation (BP) method is used to train the model and optimize the model. The result of identifying advisors is obtained through classifier after training.

4. 4.

   Select the advisor-advisee pair whose identification accuracy exceeds 90% as the advisor-advisee dataset.

The output dataset consists of 810,469 advisor-advisee pairs and more than 1,475,000 publications from 1968 to 2016, with the highest computational precision reaching 94%. While DBLP only provides publication details for each scholar, the accomplishment in terms of citations cannot be obtained from DBLP. In order to obtain scholars’ citations and h-indices, we match our dataset with AMiner-DBLP (Tang et al., 2008). AMiner ¹¹1https://www.aminer.cn is a website which aims to provide comprehensive search and mining services for researcher social networks. It is considered as a widely used and one of the best-curated databases for Computer Science articles (Gollapalli et al., 2011; Moreira et al., 2011; Tang et al., 2008; Amjad et al., 2017). Currently, the system consists of more than 6,000 conferences, 3,200,000 publications, and 700,000 researcher profiles before 2016. In this website, developers use Microsoft Graph Search API to query each AMiner paper’s title and obtain candidate matching papers for each AMiner paper. For the computational accuracy, they randomly sampled 100,000 linking pairs and evaluated the matching accuracy. The number of truly matching pairs is 99,699 and the matching accuracy can achieve 99.70%.

Finally, the analysis is on the grounds of 15,559 advisors whose academic age is 5 (23,473 advisees), 20,859 advisors whose academic age is 10 (36,841 advisees), 17,028 advisors whose academic age is 15 (33,652 advisees), 11,522 advisors whose academic age is 20 (24,883 advisees), and 7,352 advisors whose academic age is 25 (11,305 advisees). The whole process of the experiment is illustrated in Fig. 1.

We take three variables into consideration to rank a scholar: number of publications, citations, and h-indices. H-index of a scholar is calculated on the basis of the year of publications and referenced information each year. A scholar has an h-index $h$ if at least $h$ of his/her publications attract $h$ or more citations. In order to measure the advisors’ influence on advisees during different collaboration periods, the indicators for each year are calculated.

Table 1 summarizes the statistics of the advisors at the beginning of collaborating with their advisees. It illustrates the mean of advisors’ academic characteristics including the number of publications ($NP$), citations ($NC$), and h-indices ($NH$). Here we separate advisors according to their exact academic age ($AA$) when they begin guiding the advisees, to mitigate the bias of accumulated citations for advisors with different $AA$.

In order to carry out the data-based analysis of the advisor-advisee relationships, we keep a tally of relevant information of advisors and advisees in the dataset. Fig. 2 shows the range of each personal characteristic and number of advisors/advisees distribution of these characteristics.

The red points represent advisors and blue ones represent advisees. Fig. 2(a) shows the academic age distribution of advisors and advisees since their first collaboration. As shown in Fig. 2(a), most advisors’ $AA$ are 5-20 years. After that, as $AA$ increases, the number of advisors decreases. Fig. 2(c), Fig. 2(e), and Fig. 2(g), respectively illustrate the distributions of advisees (10 years after first collaborating with their advisors) and advisors (when they began mentoring a certain advisee) in terms of $NP$, $NC$, and $NH$. We only consider the advisees whose career spans at least 10 years since they first collaborated with their advisors. It can be found from Fig. 2(c), Fig. 2(e), and Fig. 2(g), that there is a clear decreasing trend in the number of scholars with the increasing of $NP$, $NC$, and $NH$. We also plot the survival function, that is $P(x>X)$ for these graphs, which will get rid of the noise and observe the differences between various groups. The results are presented in Fig. 2(b), Fig. 2(d), Fig. 2(f), and Fig. 2(h). In these figures, the horizontal axis represents scholars’ $AA$, $NP$, $NC$, and $NH$, respectively. The vertical axis is the survival rate. These survival curves have a declining trend. The steeper decline slope is on behalf of the lower survival rate. From these figures, we can see that all curves are concave. It illustrates that most scholars have relatively lower academic performance, few scholars have higher academic achievements, which can be evidenced by “Long Tail Effect” (Anderson, 2006).

Graphs in Fig. 3 are the results obtained from fitting methods for Fig. 2. Specific fitting functions are summarised in Table 2. In Table 2, $y$ represents the number of advisees/advisors and $x$ represents $AA$/$NP$/$NC$/$NH$. In order to carry out effective fitting, we use different exponential functions to determine the relationship. In fact, the existing literature (Newman, 2001) has pointed out that the degree distribution of the scientists’ networks is between the exponential and the power-law. Obviously, our results are in consistent with aforementioned law.

Fig. 4 shows the relationship between the advisors’ academic characteristics and advisees’ academic productivity as well as their impact. Academic productivity can be represented by the number of publications authored by one scholar, and academic impact can be approximated by a scholar’s h-index (Azoulay and Pierre, 2012; Owens, 2013). It is a comprehensive indicator to evaluate scholars. Fig. 4(a) shows the correlation between advisors’ $AA$ and average $NP$, $NC$, and $NH$ of their advisees 10 years after first collaborating with their advisors. It can be seen that the indicators have the same trend with the increasing of $AA$ because these indicators have a certain dependence. All of them experience an initial growth, then remain stationary, and finally reach a decline phase. When advisors’ academic ages are in the range 1-9, advisees’ academic performance shows a linear growth trend. When advisors’ academic age reaches 28, advisees’ performance shows a downward trend. In the period when advisors’ academic age is between 10 and 27, advisees’ average achievements reach the highest level and basically remain unchanged.

It is interesting to consider the possible factors underlying this phenomenon. Prior studies have noted the importance of advisors’ career coaching and social support in advisees’ development. The current study (Wang et al., 2017) finds that the productivity of scholars is dynamic over the course of their own scientific career. For scholars whose $AA<12$, the annual productivity rises up slowly. If their $AA>24$, their productivity will decline slightly with the $AA$. In the range of $12<AA<24$, their annual productivity are nearly fixed. Advisees’ average productivity/impact shows a linear growth trend because their advisors’ productivity in the range $1\leq AA\leq 9$ increases linearly. This reflects that the relationship between advisors’ career phases and advisees’ outcomes in terms of productivity and impact actually exists in the scientific mentorship. Our analyses indicate the relationship between collaborations with different $AA$ advisors and the future career performance of advisees thus need deeper exploration.

Here are some possible reasons for our findings: (a) Academic research is a long-term accumulation process. Junior advisors need time to accumulate experience in conducting research and mentoring advisees. Compared with senior ones, their capacity and resources are limited. Senior advisors have cumulative advantages, both in academic performance and resources. (b) When the academic age of advisors is at the intermediate level (in the range of 10-27 years), their academic capability may reach a certain level as well. With sufficient resources, they can provide their advisees with the most helpful guidance. In addition to that, they are also willing to strive to instruct their advisees, because their own academic performance can be improved in this way. (c) Few senior advisors have a relatively long academic career (as shown in Fig. 2(a)), that’s why senior advisors of academic age above 35 have poor performing advisees. The random fluctuation may cause this phenomenon, and thus further study is required to untangle the underlying reasons. In general, the correlation between advisees’ high performance and advisors’ academic ages is certainly reasonable, but it is not the causation of scholars’ success. Hard work and passion are still essential to scientists’ success in the scientific career.

Similarly, in Fig. 4(b), the abscissa represents the advisors’ $NH$. In this subgraph, the left vertical axis represents advisees’ average $NP$, $NC$ and right axis represents their average $NH$. Comparing with advisees’ $NC$ and $NH$, advisees’ $NP$ has the least obvious trend with the increase of advisors’ $NH$. The phenomenon elicits the truth that quantity is not equal to quality especially in academia. The maximum value of advisors’ h-index we have considered is 25 because there are few advisors with high $NH$ values (see Fig. 2). The value of indicators for advisees’ fluctuates widely along with the growth of abscissa. So the data is binned in order to make the trend more clear.

According to the influence of advisors’ academic ages on advisees, Fig. 5 indicates advisees’ h-indices whose advisors’ academic ages are 5, 10, 15, 20, 25, and the ranking of their h-indices begins at the top 5%, 5%-10%, 10%-15%, 15%-20%, 20%-25%, respectively. The polylines represent the trends of all advisees’ average h-index over time. It can be seen from the data in Fig. 5, advisors with different rankings in h-index can bring their advisees different h-indices. It is commonly perceived that high-achieving advisors literally cause their advisees higher academic productivity and impact. Selective bias is one possible reason as top advisors tend to select quality students. It is interesting to observe that as the advisors’ academic age rises, the gap among advisees narrows.

The advisees supervised by advisors whose h-index rank the top 10% always have the highest h-index. But it is not clear how significant the differences among these advisees are. On the basis of the definition of scholars’ impact, here we regard the top ten percent of advisors in h-index as $Top_{10}$ advisors. $Res$ advisors represent the rest of advisors. We try to find out the differences between advisees mentored by $Top_{10}$ advisors or $Res$ advisors. We convert the problem from a qualitative analysis to the quantitative analysis. Fig. 6 reflects advisees’ average $NP$, $NC$, and $NH$ over time since advisees first collaborated with their advisors. In Fig. 6, advisors are differentiated with academic ages and h-indices. In the figures, $T_{n}$ corresponds to $Top_{10}$ advisors whose academic ages are $n$ while $R_{n}$ corresponds to $Res$ advisors whose academic ages are $n$. From the figures, we can see that advisors’ $NP$ results in the least obvious discrepancy between advisees supervised by $Top_{10}$ advisors and $Res$ advisors. The most striking observation to emerge from the data comparison is the growth rate of each indicator. Advisees mentored by $Top_{10}$ advisors have higher $NP$, $NC$, and $NH$ than the others. Moreover, their growth rate is higher. This result may be explained by the Matthew Effect (Langfeldt et al., 2015). In academia, while excellent scholars’ accomplishments and reputation tend to snowball, those with modest accomplishments have greater difficulty to improve their impact.

In Fig. 7, similarly, which is consistent with assessment standards of advisors, the advisees whose h-index rank the top 10% of each indicator are considered as the $Top_{10}$ advisees. The differences between $Top_{10}$ advisees’ $NP$, $NC$, and $NH$, who are mentored by $Top_{10}$ advisors or $Res$ advisors are highlighted in Fig. 7(d), Fig. 7(e), and Fig. 7(f). To the same extent as the previous results, comparing with $NC$ and $NH$, the discrepancy between advisees’ average $NP$ mentored by $Top_{10}$ advisors or $Res$ advisors is not obvious. However, considering academic ages of the advisors, $Top_{10}$ advisees mentored by $Top_{10}$ advisors whose academic age is 30 have the lowest average $NH$. For $Res$ advisors, the same scenario occurs to advisors whose academic ages are 5. It is different from advisors’ $NC$ and $NH$. This phenomenon gives us inspiration that only considering the number of publications to evaluate scholars is insufficient because quantity is not always equal to quality. As a result, a number of comprehensive indices have appeared to evaluate scholars. In Fig. 6 and Fig. 7, it can be seen from the subgraphs that comparing with all of the advisees, $Top_{10}$ advisees mentored by $Top_{10}$ advisors or $Res$ advisors make more distinct values in the average $NP$, $NC$, and $NH$.

The results of the correlational analysis among advisor-advisee’s collaboration duration, advisors’ academic ages, proportion of advisees’ h-index ranking the top 10% are shown in Fig. 8. The data in first three years of collaboration with advisors whose academic age is 5 are missing because of the slow growth rate in this period. It is difficult to calculate the top 10% advisees’ h-indices because the values of their h-indices are relatively low and similar. For $Res$ advisors, the proportion of advisees ranking top 10% is shown in the left subgraph in Fig. 8(a), and the result of $Top_{10}$ advisors is summarized in the right subgraph. Fig. 8(b) shows the proportion of differences between advisees mentored by $Top_{10}$ advisors and $Res$ advisors. The values of advisors whose academic ages are 30 in Fig. 8(b) are always below 0.1. For others, their values are all above 0.1, even the maximum value can reach 0.28. It indicates that most $Top_{10}$ advisees are coached by $Top_{10}$ advisors.

Selective bias is a possible explanation for these phenomena. The advisor-advisee relationship is a two-way choice relationship, quality advisees tend to seek top advisors and vice versa. Moreover, we have also found that the gap between these advisees is closely related to the advisors’ academic age and the time duration of their collaboration. In the first three years of collaboration, the gap between advisees is relatively small. Since the fourth year, the gap begins to grow wider.

As mentioned before, we have separated advisors according to their $AA$ in the experiment. However, considering the publication rates increased over time, it may have a biased effect on the scholars’ number of publications, citations, and h-indices. So we carry out the experiment to verify whether the publications rates impact our findings.

Fig. 9(a) displays the number of new entries in the database per year. From the data in Fig. 9(a), it is apparent that the growth rate of the publication is different at different stages. The number of new records increases sharply in 2003 and 2011. To be more specific, there are less than 25,000 new publications per year in 1999-2002 but over 50,000 in 2003-2007, almost two times more than the first period. In order to ensure all of the advisees have plenty of time to accumulate enough publications, we choose advisors whose academic age is 20 in 1996-2002 and 2003-2007, respectively. We compare the average academic performance (h-index) of their advisees. In Fig. 9(b) and Fig. 9(c), the polylines show the advisees’ average h-index mentored by different h-index ranking advisors. Data in these figures can be compared with the data in Fig. 5, which is the basis of our findings. A positive correlation can be found between advisors’ and advisees’ academic performance. So the publication rates have limited influence on the phenomena we have found in this work.