Characterizing Sociolinguistic Variation in the Competing Vaccination Communities

To be able to conduct any analysis, it is imperative to identify the competing groups. Assigning a stance to a tweet or a twitter user is a non-trivial problem. Therefore, we use a similar method as described in [22, 23] to find anti-vaxxers and pro-vaxxer groups based on the weighted combination of the valence of their hashtags. In this study, we assume that retweets indicate endorsement.

In the previous studies such as [11], hashtags have been shown to work as realistic proxies for identifying stances among different groups on social media sites. In [22], hashtags are used to identify twitter users who believe in anthropogenic causes of climate change and those who do not. Similarly, in [23], hashtags could also be used to identify polarization in political discourse and how the polarization can change with time.

We use community detection method based on the work done in [23]. We first choose 2 seed hashtags for each of the polarized groups: #VaccinesSaveLives and #VaccinesWork for pro-vaccination and #VaccineInjury and #LearnTheRisk for anti-vaccination. We assign pro-vaccination seeds a valence of +1, and anti-vaccination seeds a valence of -1. ¹¹1We randomly sample 100 tweets for each of these hashtags. For pro-vaccination hashtags, $98\%$ of tweets with hashtag #VaccinesSaveLives and $97\%$ of tweets with hashtag #VaccinesWork were related to pro-vaccination. For anti-vaccination hashtags, $88\%$ of tweets with hashtag #LearnTheRisk and $93\%$ of tweets with hashtag #VaccineInjury were related to anti-vaccination. We then create a hashtag co-occurrence graph to identify most co-occurring hashtags with the chosen seeds, and choose those that are semantically similar, as well as the ones that are known to be pro-vax and anti-vax hashtags from the background literature [9, 5, 4] to manually assign a hard valence of +1 and -1. We then use a variant of label propagation algorithm [25] described as Algorithm 1 below to assign valence to each of the remaining hashtags. Similar to [23] the input to the algorithm is a hashtag-to-hashtag co-occurrence graph where hashtags represent nodes, and nodes are connected if they co-occur. The edges are weighted by the frequency of co-occurrence.

Once we have identified the valence of a set of hashtags, we aggregate hashtags used by each user and find a weighted average of the valence of all hashtags used by a particular user. We label a user as pro-vaxxer, or anti-vaxxer if the weighted average was positive, or negative respectively.

Using the algorithm, 3295 users are identified as pro-vaxxers, 2967 as anti-vaxxers. We randomly sample 100 users that were classified as pro-vaxxers and 100 users that were classified as anti-vaxxers to evaluate the quality of assignment. We find $96\%$ of the labeled pro-vaxxers as pro-vaxxers, and $80\%$ of the labeled anti-vaxxers as anti-vaxxers.

We first study the differences in the usage of linguistic intensifiers. Intensifiers are words, or phrases that strengthen the meaning of other expressions and show emphasis. Examples include amplifiers (eg.“really”, “very”), usage of swear words, general interjections (eg. “wow”, “omg”), and exclamations. Intensifiers are commonly used to bolster argumentation to persuade the target audience. We hypothesize that users that are pro-vaxxers use more intensifiers. This is because pro-vaxxers have been found to frequently debunk anti-vaxxers’ claims with scientific evidence [2]. Therefore, they would seem to take the corrective approach intended to persuade anti-vaxxers, hence using more intensifiers.

Pronouns play a key role in models of narrative and discourse processing [12]. Because most of the vaccine-related misinformation is based on personal anecdotes, we would expect pronominal usage to be high amongst anti-vaxxers. To test this, we identify various different categories of pronouns (eg. “subject pronouns”, “object pronouns”, “third-person pronouns”), a complete list of which can be found in Table 2.

Previous research [10] has found the use of uncertainty words (eg. “might”, “likely”) as a negative linguistic correlate of echo-chamberness. This is based on the hypothesis that because users not in echo-chambers are exposed to alternate views, they may be less certain of their ideas. We adopted the list of uncertainty words from [10] to test if that is true i.e. if there is a significant difference in the use of uncertainty words across the two vaccination communities.

Network density is defined as the ratio of actual connections and potential connections [13]. Dense networks tend to “groupthink” [21] where conformity of ideas is highly valued and difference of opinions is discouraged.

The EI (External-Internal) index was developed by Krackhardt and Stern in [17] as a measure of dominance of external over internal ties. More concretely, assuming two groups based on some attribute, one group defined as internal and the other as external, the EI index is computed as follows:

where EL represents the number of external links and IL represents the number of internal links. EI index is a useful proxy for identifying echo-chamberness.

To compare the echo-chamber effect in the two vaccination groups, we also directly compute the echo-chamberness of the two communities. We use the following definition of echo-chamberness: For a given network $G$, the echo-chamberness (EC) is defined as:

where r is the reciprocity [24] of graph G or the ratio of bi-directional edges and the total number of edges in G, and d is the density of graph G.

For each sub-category of the linguistic features in Table 2, we use two test statistics to compute the difference between the two groups. These are as follows:

1. 1.

   The overall proportion of tweets that contain any of the words for a given lexical category ($T_{1}$)

2. 2.

   The mean of the proportions of tweets of individual users containing any of the words for a given lexical category ($T_{2}$)

We use these test statistics to compute (i) the difference of proportions between the two groups, and (ii) the difference of means of proportions between the two groups.

The first test statistic regards each tweet independently. We use the second test statistic to account for differences in the linguistic choices of individual users.

For the first statistic, we use a two-sample z-test for the difference of proportions ($Z_{1}$). For the second statistic, we use an independent z-test for the difference in means ($Z_{2}$). For all the tests, our $\alpha=0.05$.

We observe that our initial hypothesis that pro-vaxxers use more intensifiers is false. What we find is that anti-vaxxers employ significantly more linguistic intensifiers than pro-vaxxers. This holds true across all the sub-categories of intensifiers with the exception of the use of general interjections where the difference is marginal and not significant. While intensifiers are used as a persuasion technique, the observed results can possibly be explained by an old theory in speech communication that correlates the use of intensifiers with perceived powerlessness [3, 14]. Intensifiers and hedges are used more generally by people with low social power [3]. Because anti-vaxxers are a minority group, it is a possible argument one could make as perceived minority leads to perceived low social power which could lead to high linguistic intensification.

From our analyses, we find that with the exception of reflexive and possessive pronouns, anti-vaxxers show a significantly high pronominal usage across all the categories. This difference is prominent specifically for third-person, gendered third-person, subject, and object pronouns. In sociolinguistic literature, pronouns are predominantly linked with narrative discourse structure. For example object pronouns such as “him” or “his” and gendered third-person pronouns “he” or “she” have a referential property, where their semantic interpretation is dependent on what they are referring to. Anaphoric references define objects already defined in the discourse [26] which creates a better narrative viewpoint. Like intensifiers, pronouns are also found to be used heavily by people with lower levels of perceived power [19].

In terms of the use of uncertainty words, while we do find a significant difference between the two communities, we do not observe the same effect observed in the background literature [10]. In fact, we find a counter-intuitive result i.e. that the anti-vaccination community with higher echo-chamberness (as observed in section 4.2) tends to use more uncertain words than pro-vaccination community. This is an evidence that not all echo-chamber communities show certainty in their tweets as observed in [10].