We curated a keyword list for each of the three controversial topics – Abortion, Gun Control, and BLM. Keywords for Abortion and Gun Control came from another recent work (guo2020inflating) and were expanded using contextual expansion with sample tweets. Keywords for BLM were manually curated, separating into pro- and anti-BLM keywords respectively. We expanded the latter with white-supremacy related keywords. Table 3 lists all keywords.

We consider a video is potentially relevant if (a) it contains at least one keyword in the video title or description; or (b) it is mentioned in a tweet that contains the keywords in the tweet text. We obtained 815 candidate videos for Abortion, 974 candidate videos for Gun Control, and 8,571 candidate videos for BLM.

For Abortion and Gun Control, we manually annotated all candidate videos. All videos were randomly divided into five buckets (with 163 or 195 videos in each) and each bucket was labeled by an annotator. The five authors of this paper, along with two postgraduate students who had extensive research experience in computational social science, consisted of the annotator team. Annotators were instructed to watch a video for at least five minutes before making a decision of whether or not this video is relevant to the target topic. To measure the inter-rater reliability, we randomly selected a bucket, and had two more annotators labeling the same bucket. We computed the Fleiss’ Kappa. Abortion had Fleiss’ Kappa of 0.830 and Gun Control had that of 0.826, suggesting a high agreement among annotators.

For BLM, we used a semi-automatic approach because the number of candidate videos (8,571) is much larger. Based on our observations on Abortion and Gun Control, we designed the following protocol:

• •

  For videos that contained any topic-relevant keywords in the video titles or descriptions,

  • –

    if the videos were also mentioned by tweets with topic-relevant keywords, 89.0%/86.7% of the videos were labeled relevant (in Abortion and Gun Control respectively, same below). These videos consisted of 16.1%/12.5% of all candidate videos. We labeled such videos as relevant, there were 418 in BLM.

  • –

    if the videos were not mentioned by tweets with topic-relevant keywords, 77.8%/89.4% of the videos were labeled irrelevant. These videos consisted of 6.7%/11.9% of all candidate videos. We labeled such videos as irrelevant.

• •

  For videos that did not contain any topic-relevant keywords in the titles nor descriptions, but had been mentioned by some topic-relevant tweets,

  • –

    if the videos were tagged as “Music”, or did not have English captions, or did not contain any topic-relevant keywords in the captions, 95.2%/80.6% of the videos were labeled irrelevant. We labeled such videos as irrelevant.

  • –

    we used the manual annotation steps for Abortion and Gun Control to label the remaining 749 videos, out of which 359 videos were labeled relevant. The Fleiss’ Kappa was 0.711.

In total, we identified 179 Abortion videos, 268 Gun Control videos, and 777 BLM videos. We extracted all tweets and Twitter users that mentioned any of these topic-relevant videos from our 16 months data stream.

We estimated the political leanings of early adopters on Twitter in four steps with increasing coverage.

The shared audience network is shown to be an effective method for splitting users with different political behaviors (Stewart et al. 2017). When estimating the user ideologies, our assumption is that users with the same ideology share similar audience. The shared audience network was a dense network since we created edges between any two users regardless of whether they had a direct follower-following relation. The initial networks consisted of approximately $28M$, $119M$, and $394M$ edges for Abortion, Gun Control, and BLM, respectively. The corresponding network densities were 18.1%, 15.2%, and 15.7%. To reduce the network complexity, we used a disparity filtering algorithm (serrano2009extracting) to extract the network backbone. The disparity filtering algorithm can identify important edges in a network. It has a hyperparameter $\alpha$ to control the degree of edge importance. We used $\alpha=0.05$ as suggested in the initial work (serrano2009extracting). After disparity filtering, the resulting networks contained approximately $1.9M$, $7.8M$, and $25.5M$ edges for Abortion, Gun Control, and BLM, respectively. The densities of these networks ranged between 1% and 1.2%.

Next, we applied a label propagation algorithm (Zhou et al. 2004) on the filtered network to infer the political leanings of the unknown users from those of the seed users. Label propagation is a semi-supervised algorithm that builds on two intuitions: (a) the seed labels should not change too much after propagation; (b) the inferred labels should not be too different with their neighbours. A teleport factor $\beta$ controls the trade-off between these two intuitions. We used $\beta=0.85$ as suggested in the prior work (yan2020mimicprop). The label propagation algorithm returned a liberal score and a conservative score for each user in the network, which we then re-normalized into a score between 0 and 1, where 0 indicated strong liberal and 1 indicated strong conservative. Re-normalization was done by computing the ratio of conservative score to the summation of liberal and conservative scores.

We stratified one-tenth of seed users in each fold and hided their labels. Next, we compared the inferred political leanings to actual leanings of these users. Note that we employed whole network during the label propagation validation. Table 6 shows the evaluation results for the label propagation.

Once we obtained the ideology probabilities of early adopters on Twitter, we computed the leaning score for each YouTube video by averaging its promoters’ ideology scores. To assign discrete leaning labels (i.e. Left, Right and Center) to the videos, we leveraged the Recfluence dataset (Ledwich and Zaitsev 2020) as an external source to annotate YouTube videos’ leanings. This dataset includes 816 YouTube channels where each channel has 10k+ subscribers and more than 30% of its content is relevant to US politics, or cultural news, or cultural commentary. It consists of channels of YouTubers in addition to those of traditional news outlets. In this dataset, each channel is assigned to Left (L), Center (C) or Right (R) leaning. The annotations were conducted based on two well known media bias resources⁴⁴4Media Bias/Fact Check: https://mediabiasfactcheck.com/; Ad Fontes Media: https://www.adfontesmedia.com/ as well as manual labeling.

The videos whose channels’ leanings are available in the Recfluence dataset are assigned to the same leaning. We were able to label 109 videos (L: 38, R: 66, C:5) from 38 channels for Abortion, 153 videos (L: 52, R: 88, C: 13) from 62 channels for Gun Control, and 521 videos (L: 194, R: 285, C:42) from 120 channels for BLM. To label the videos whose channels are not available in the Recfluence dataset, we identified two thresholds for each topic – $thr_{(L,C)}$ and $thr_{(C,R)}$ where $thr_{(L,C)}<thr_{(C,R)}$. For a given video $\nu$, we assigned a discrete leaning label, $leaning\_label(\nu)$, based on its leaning score, $leaning\_score(\nu)$, as follows:

We used videos with labels from Recfluence to find optimum thresholds $thr_{(L,C)}$ and $thr_{(C,R)}$. We obtained a leaning score distribution for each group (i.e., L, R and C). We identified outliers for each group using inter-quartile rule and filtered them out. The leaning score distributions by groups for each topic are given in Figure 8. Next, we found the thresholds $thr_{(L,C)}$ and $thr_{(C,R)}$ using posterior probabilities as follows:

The resulting threshold pairs ($thr_{(L,C)}$, $thr_{(C,R)}$) are $(0.538,0.749)$ for Abortion, $(0.478,0.716)$ for Gun Control and $(0.524,0.695)$ for BLM, respectively. Based on these thresholds, we inferred the leaning labels of 70 videos (L: 20, R: 45, C: 5) from 58 channels for Abortion, 115 videos (L: 29, R: 66, C: 20) from 98 channels for Gun Control, 256 videos (L: 103, R: 111, C: 42) from 197 channels for BLM.

To validate the video leaning estimation, we conducted a manual leaning annotation for videos in Gun Control. We used stratified sampling where we sampled 5 videos from each 10 percentile bin of their leaning scores. The leaning of these 50 videos were annotated independently by three authors considering three categories (left-leaning, right-leaning, unknown). The Fleiss’ Kappa is 0.691, suggesting a moderate agreement among annotators. We used the majority vote to assign a video label. Table 7 summarizes the annotation results.

We observed that the only decile bin with both manually annotated left- and right-leaning video is the 30th-40th percentile of leaning scores $(0.465-0.665)$. Suggesting that the annotator had high agreement with the estimated leaning scores, and that the threshold should be in this interval – which is indeed the case for Gun Control with $thr_{(L,C)}$ lies within its left boundary, and $thr_{(C,R)}$ only 0.05 away from its right boundary. In addition, most of the videos which were assigned unknown by the raters are the footage of shooting of Philando Castile and mass shooting events including Las Vegas shooting and TX massacre. These videos do not include any stance on their contents. However, we observed that the leaning scores of these videos align well with the leaning of channels. One example is “Comments on TX church shooting” where the video leaning score belongs to the second bin $(80th-90th)$ percentile. The channel owner is a far-right YouTuber. The video leaning score matches with the channel owner’s leaning, our approach is thus successful for solving such ambiguities.

Table 7 shows manual leaning annotation results of 50 Gun Control videos. Compared to the leaning score range in the second column (which is estimated), the first three rows of 14 videos (which are labeled as left-leaning) are consistently labeled (the result of manual labeling and estimated leanings agree) as well as the last five rows of 19 videos (which are labeled as right-leaning). The computed thresholds for Gun Control videos, as presented in Figure 8, are $0.478$ and $0.716$.

From the collected videos we extracted topic (Abortion, Gun Control and BLM) relevant videos for our topical analysis, but all collected videos were used to compute relative engagement scores. We filter out videos that receive less than 100 views within their first 30 days after upload, which is the same filter used by brodersen2012youtube. We summarize how relative engagement score is computed. For a given video, first we compute two aggregate metrics:

• •

  average watch time ($\bar{\omega}_{t}$): the total watch time $x_{w}[1:t]$ divided by the total view count $x_{v}[1:t]$ up to day $t$

  $$\bar{\omega}_{t}=\frac{\Sigma^{t}_{i=1}x_{w}[i]}{\Sigma^{t}_{i=1}x_{v}[i]}$$

• •

  average watch percentage ($\bar{\mu}_{t}$): the average watch time $\bar{\omega}_{t}$ normalized by video duration D

  $$\bar{\mu}_{t}=\frac{\bar{\omega}_{t}}{D}$$

Then Engagement map of tweeted videos is constructed as follows. Two maps are created; the first where x-axis shows video duration D, and the y-axis shows average watch time over the first 120 days ($\bar{\omega}_{120}$) (Figure 9(a)) and the second with the same axis with average watch percentage over the first 120 days ($\bar{\mu}_{120}$) (Figure 9(b)) as y-axis. All videos in the tweeted videos dataset are projected onto both maps. The x-axis is split into 1,000 equally wide bins in log scale. These 2 maps are logically identical but Figure 9(b) is easier to read as y-axis is bounded between [0,1]. This second map is denoted as Engagement map.

Based on this Engagement map, relative engagement $\bar{\eta}_{t}\in[0,1]$ defined as the rank percentile of video in its duration bin. This is an average engagement measure in the first t days. Figure 9(b) illustrates the relation between video duration D, watch percentage $\bar{\mu}_{120}$ and relative engagement $\bar{\eta}_{120}$ for three example videos. $V_{1}$ is a left-leaning video in Abortion topic with duration of 136 seconds. $V_{2}$ is a right-leaning video in Abortion topic with duration of 141 seconds. $V_{1}$ and $V_{2}$ have similar lengths but different watch percentages, $\bar{\mu}_{120}(V_{1})=0.83$ and $\bar{\mu}_{120}(V_{2})=0.59$. The difference becomes more apparent with relative engagement scores, $\bar{\eta}_{120}(V_{1})=0.94$ and $\bar{\eta}_{120}(V_{2})=0.27$ as shown in Figure 9(b). $V_{3}$ is also a right-leaning video in Abortion which is significantly long than the others. Although $V_{3}$ is not as much watched as the previous two, $\bar{\mu}_{120}(V_{3})=0.40$, relative engagement of $V_{3}$ is much higher than $V_{2}$, $\bar{\eta}_{120}(V_{3})=0.86$ as longer videos tend to have lower average watch percentages.

Hawkes Intensity Process (HIP) (Rizoiu et al. 2017) extends the well-known Hawkes (self-exciting) process (hawkes1971spectra) to describe the volume of activities within a fixed time interval (e.g. daily). This is done by taking expectations over the stochastic event history. Specifically, this model describes a self-exciting phenomenon that is commonly observed in online social network (Rizoiu and Xie 2017). It models the target quantity $x[t]$ as a self-consistent equation into three parts: the unobserved external influence, the effects of external promotions, and the influence from historical events. Formally, it can be written as

The first two terms represent unobserved external influences. $\gamma$ and $\eta$ model the strengths of an initial impulse and a constant background rate, respectively. In the middle component, $\alpha$ is the sensitivity to external promotion, $s[t]$ is the volume of promotion, and $\alpha s(t)$ is the instantaneous response to promotion. In the last component, $\theta$ is the exponent of a power-law memory kernel $(\tau+c)^{-(1+\theta)}$. $c$ is a nuisance parameter for keeping the kernel bounded, and $C$ accounts for the latent content quality. Overall, this last component models the impact over its own event history $x[\tau]$ for $\tau{=}1:t{-}1$.

In our case, $x[t]$ is the time series of daily views $x_{v}[t]$ on YouTube. $s[t]$ is the daily number of tweets on Twitter. The parameter set $\{\gamma,\eta,\alpha,C,c,\theta\}$ is estimated from the first 90-day interval of each video using the constrained L-BFGS algorithm in SciPy Python package.⁵⁵5https://github.com/andrei-rizoiu/hip-popularity

Figure 10 and Figure 11 visualize the attention accumulation for Gun Control and BLM topic.

Figure 12 visualizes attention distribution of the isolated, small, and large cascades for Gun Control and BLM topic.

We measured additional metrics on early-adopter networks, the comparative findings are summarized in Table 2.

We link the measure of relative engagement of YouTube videos to the number of tweets, the number of followers, and the number of views. The number of tweets and views are accumulated for the first 120 days after upload. Rather than looking at the overall correlation, which is dominated by large variations, we choose to group videos according to relative engagement. This allows us to interpret the change in Twitter user base, tweeting behavior, and YouTube attention across videos of different engagement levels.

We compute relative engagement score for each video and rank the videos in descending order of relative engagement score. Then, the videos are divided into 5 groups, each including the top 20% videos, top 20%~40% videos, top 40%~60% videos, top 60%~80% videos and top 80%~100% videos. For each group, left and right-leaning videos are shown as boxplots of Tweet120, Follower count, and View120 (Figure 13).

As seen earlier, left-leaning videos have less number of tweets on average which can be seen with Tweet120. Especially, in the case of Abortion, the middle buckets (20%, 40% and 60%) show that left-leaning videos have significantly less tweets but have more views on average. The same pattern holds for Gun Control. This is consistent with the previous finding that left-leaning videos are more effectively promoted (higher viral potential) with fewer number of tweets.

We can observe that higher numbers of followers does not imply higher numbers of tweets on average as witnessed by most of middle buckets. However, we note that the top three outliers in left-leaning videos in 40% bucket of Follower count boxplot for Gun Control corresponds to the top three outliers of left-leaning videos in the same bucket of Tweet120.

In terms of View120, the average trends support findings in Section 5.1 in that left-leaning videos in Abortion and Gun Control have higher views whereas left-leaning videos in BLM have lower views. Although view count is considered as an important measure of video popularity in general, no noticeable correlation is shown between relative engagement and View120. This is expected as engagement metric and popularity metric capture very different perspective of online attention (wu2019estimating). \nociteAP* \bibliographystyleAPaaai22 \bibliographyAPxplatform-ap-ref