A General Method to Find Highly Coordinating Communities in Social Media through Inferred Interaction Links

Temporal information is a key element of coordination, and thus is critical for effective coordination detection. Frequent posts within a short period may represent genuine discussion or deliberate attempts to game trend algorithms (Grimme et al., 2018; Varol et al., 2017; Assenmacher et al., 2020). We treat the post stream as a series of discrete windows to constrain detection periods. An LCN is constructed from each window (Step 4), and these are then aggregated and mined for HCCs (Step 5). As we assume posts arrive in order, their timestamp metadata can be used to sort and assign them to windows.

Figure 3 gives an example of searching for co-hashtag coordination across Facebook, Twitter, and Tumblr posts. The posts are converted to interaction primitives in Step 1, shown in Figure 3a. The information required from each post is the identity of the post’s author⁵⁵5Linking identities across social media platforms is beyond the scope of this work, but the interested reader is referred to Adjali et al. (2020) for a recent contribution to the subject., the timestamp of the post for addressing the temporal aspect, and the hashtag mentioned (there may be many, resulting in separate records for each). This is done in Figure 3b, which shows the filtered mentions (in orange) and hashtag uses (in purple), ordered according to timestamp.

Step 3 in Figure 3c involves searching for evidence of coordination through searching for our target coordination strategies through pairwise examination of accounts and their interactions. Here, three accounts co-use a hashtag while only two of them co-mention another account.

By Step 4 in Figure 3d, the entire LCN has been constructed, and then Figure 3e shows its most highly coordinating communities.

As mentioned above, to account for the temporal aspect, the LCNs produced for each time window in Figure 3d can be aggregated and then mined for HCCs, or HCCs could be extracted from each window’s LCN and then they can be aggregated, or analysed in near real-time, as dictated by the application domain.

In addition to relevant datasets, we make use of a ground truth (GT), in which we expect to find coordination (cf., Keller et al., 2017; Vargas et al., 2020). By comparing the evidence of coordination (i.e., HCCs) we find within the ground truth with the coordination we find in the other datasets, we can develop confidence that: a) our method finds coordination where we expect to find it (in the ground truth); and b) our method also finds coordination of a similar type where it was not certain to exist. Furthermore, to represent the broader population (which is not expected to exhibit coordination), similar to Cao et al. (2015), we create a randomised HCC network from the non-HCC accounts in a given dataset, and then compare its HCCs with the HCCs that had been discovered by our method.

Our primary factors include the HCC extraction method (using FSA_V, $kNN$, or thresholds), the window size, $\gamma$, and the strategy being targeted (Boost, Pollution or Bully). Given our interest prioritises the sets of accounts over how they are connected, for each pair of variations we compare the membership of the HCCs discovered, as a whole, and the number of HCCs discovered, as well as the edge count. Also included are the exact numbers of HCC members common to each pair. These figures provide further context for the degree of overlap between the HCC members identified under different conditions (i.e., factor values). We use Jaccard and overlap similarity measures (Verma and Aggarwal, 2020) to compare the accounts appearing in each (ignoring their groupings) and render them as heatmaps. The Jaccard similarity coefficient, $J$, is a value between $0.0$ and $1.0$ which represents the similarity between two sets of items, $X$ and $Y$:

If there is significant imbalance in the sizes of $X$ and $Y$, then their similarity may be low, even if one is a subset of the other. An alternative measure, the Overlap or Szymkiewicz-Simpson coefficient (Verma and Aggarwal, 2020), takes this imbalance into account by using the size of the smaller of the two sets as the denominator:

In a circumstance such as ours, it is unclear whether a longer time window will garner more results after HCC extraction is applied. The Jaccard and overlap coefficients can be used to quickly understand two facts about the sets of accounts identified as HCC members with different values of $\gamma$:

• •

  Is one set a subset of the other? If so, the overlap coefficient will reach $1.0$, while the Jaccard coefficient may not, if the two sets differ in size. If they are disjoint, it will be $0.0$, along with the Jaccard coefficient.

• •

  Do the sets differ in size? If the sets are different sizes, but one is a subset of the other, the overlap coefficient will hide this fact, while the Jaccard coefficient will expose it. If both coefficients have values close to $0.0$, then the sets are clearly different in membership and potentially also in size. If the coefficient values are very close, then the sets are close in size, because the denominators are similar in size, meaning $|X\cup Y|=~{}min(|X|,|Y|)$, but this will only occur if they share many members (i.e., $|X\cap Y|$ is high).

The exact numbers of common accounts complement the degree of overlap visible in the heat maps to provide the reader a basis to easily compare how different variations compare with each other beyond examining them only as pairs and therefore illuminates the influence each particular factor has overall. For example, by being able to compare the results using each value for $\gamma$, it is possible to see the progression of results as the window size increases (both in raw numbers and with a colour scale in the heatmaps).

A second subjective method of analysis for networks is, of course, to visualise them. We use two visualisation tools, visone (https://visone.info) and Gephi (https://gephi.org), both of which make use of force directed layouts, which help to clarify clusters in the network structure. In particular, Gephi provides access to the Fruchterman-Reingold (FR) algorithm (Fruchterman and Reingold, 1991), which is force directed, but constrains the layout within a circle. This enables us to see to what degree individual clusters dominate the overall graph, in a way that standard force-directed layouts do not. Node colour is used to represent cluster membership. Clusters are identified with the Louvain method (Blondel et al., 2008). Node size can be used to represent degree or, for nodes representing accounts, the number of posts an account contributes to a corpus. Edge thickness and colour (darkness) are used to represent the weight of the edges. As HCCs are weighted undirected networks and each connected component is an HCC, node colour can be used to represent the number of posts, and edge weight can be represented by thickness and, depending on the density of the network, colour darkness. Node shape can be also employed to represent a variable; for analyses that involve multiple criteria (e.g., co-conv and co-mention), we use shape to represent which combination of criteria an HCC is bound by (e.g., just co-mention or a combination of co-mention and co-conv or just co-conv).

By extending the HCC account networks with nodes to represent the ‘reasons’ or instances of evidence that link each pair of nodes, e.g., the tweets they retweet in common, or accounts they both mention or reply to, creating a two-level account-reason network in doing so, we can investigate how HCCs relate to one another. In this case, the two-level network has two types of nodes (accounts and reasons) and two types of edges (‘coordinates with’ links between accounts and ‘caused by’⁶⁶6Or other appropriate phrasing to indicate that the account is linked to the reason, by which it is indirectly linked to other accounts. N.B. The reason itself may be an account, e.g., in the context of co-mentions, but for the purpose of the two-level network, the mentioned account is treated as a reason. links between ‘reasons’ and accounts). Visualising the two-level network by colouring nodes by their HCC and using a force-directed layout highlights how closely associated HCCs are with each other, not only revealing what reasons draw many HCCs together (i.e., many HCCs may be bound by a single reason, or an HCC may be entirely isolated from others in the broader community), but also how many reasons may bind them (i.e., many reasons may bind an HCC together). Deeper insights can be revealed from this point using multi-layer network analyses.

To help answer RQ2, it is helpful to look beyond network structures and consider how consistent the content within an HCC⁷⁷7I.e., the content produced by the HCC’s members. is relative to other HCCs and the population in general. This will be most applicable when the type of strategy the HCC is suspected to have engaged in requires repetition, e.g., co-retweeting or copypasta. If HCCs are boosting a message, it is reasonable to assume the content of HCCs members will be more similar internally that when compared externally, to the content of non-members. To analyse this internal consistency of content, we treat each HCC member’s tweets as a single document and create a doc-term matrix using $5$ character n-grams for terms. Comparing the members’ document vectors using cosine similarity in a pairwise fashion creates a $n\cdot n$ matrix where $n$ is the number of accounts in the HCC network. This approach was chosen for its performance with non-English corpora (Damashek, 1995), and because using individual tweets as documents produced too sparse a matrix in a number of tests we conducted. The pairwise account similarity matrix can be visualised, using a range of colours to represent similarity. By ordering the accounts on both the $x$ and $y$ axes to ensure they are grouped within their HCCs, if our hypothesis is right that similarity within HCCs is higher than outside, then we should observe clear bright squares extending from the diagonal of the matrix. The diagonal itself will be the brightest because it represents each account’s similarity with itself.

If HCCs contribute few posts, which are similar or identical to other HCCs, then bright squares may appear off the diagonal, and this would be evidence similar to clusters of account nodes around a small number of reason nodes in the two-level account-reason networks mentioned in the previous section.

This method offers no indication of how active each HCC or HCC member is, so displays of high similarity may imply low levels of activity as well as high content similarity, just because of the lower likelihood that highly active accounts are highly similar in content (by contributing more posts, there are simply more opportunities for accounts’ content to diverge). The use of the 5-character n-gram approach is designed to offset this because each tweet in common between two accounts will yield a large number of points of similarity, as will the case when the same two tweets are posted in the same order (i.e., two accounts both post tweet $t_{1}$ and then $t_{2}$), because the overlap between the tweets will yield at least 4 points of similarity.

Converse to the consistency of content within HCC is the question of content variation, and how does the variation observed in detected HCCs differ from that of RANDOM groupings. Highly coordinated behaviour such as co-retweeting involves reusing the same content frequently, resulting in low feature variation (e.g., hashtags, URLs, mentioned accounts), which can be measured as entropy (Cao et al., 2015). A frequency distribution of each HCC’s use of each feature type is used to calculate each entropy score. Low feature variation corresponds to low entropy values. As per Cao et al. (2015), we compare the entropy of features used by detected HCCs to RANDOM ones and visualise their cumulative frequency. Entries for HCCs which did not use a particular feature are omitted, as their scores would inflate the number of groups with $0$ entropy.

Hashtags can be used to define discussion groups or chat channels (Woolley, 2016), so hashtag analysis can be used to study those communities. It is another aspect to content analysis that relies upon social media users declaring the topic of their post through the hashtags they include. At the minimum, we can plot the frequency of the most frequently used hashtags as used by the most active HCCs. In doing so, we can quickly see which hashtags different HCCs make use of, and how they relate by how they overlap. Some hashtags will be unique to HCCs, while others will be used by many. This exposes the nature of HCC behaviour: they may focus their effort on a single hashtag, perhaps to get it trending, or they may use many hashtags together, perhaps to spread their message to different communities.

To further explore how hashtags are used together, we perform hashtag co-occurrence analysis, creating networks of hashtags linked when they are mentioned in the same tweet. These hashtag co-occurrence networks are sometimes referred to as semantic networks (Radicioni et al., 2020). When visualised it is possible to see themes in the groupings of hashtags, and to gain insights from how the theme clusters are connected (including when they are isolated from one another). To extend this approach, Louvain clustering (Blondel et al., 2008) can be applied and hashtag nodes can be coloured by cluster, providing a statistical measure to how related the hashtags are.

Campaign types can exhibit different temporal patterns (Lee et al., 2013), so we use the same temporal averaging technique as Lee et al. (2013) (dynamic time warping barycenter averaging) to compare the daily activities of the HCCs in the GT and RANDOM datasets with those in the test datasets. The temporal averaging technique produces a single time series made from the time series of all activities of each member in a set of accounts. Using this technique avoids averaging out of time series that are off-phase from one another, by aligning them before averaging them.

Another aspect of temporal analysis is the comparison of HCCs detected in different time windows, including specifically observing whether any HCCs between time windows share members and what the implications are for the behaviour of those members. This is non-trivial for any moderately large dataset, but examination of the ground truth can provide some insight into the behaviours exhibited by accounts known to be coordinating.

Groups that retweet or mention themselves create direct connections between their members, meaning if one is discovered, it may be trivial to find its collaborators. To be covert, therefore, it would be sensible to have a low internal retweet and mention ratios (IRR and IMR, respectively). Formally, if $RT_{int}$ and $M_{int}$ are the the sets of retweets and mentions of accounts within an HCC, respectively, and $RT_{ext}$ and $M_{ext}$ are the corresponding sets of retweets and mentions of accounts outside the HCC, then, for a single HCC,

and

The method presented Section 3.1 highlights HCCs that coordinate their activity at a high level over an entire collection period. Further steps can be taken to determine which HCCs are coordinating their behaviour repeatedly and consistently across adjacent time windows. In this case, for each time window, we consider not just the nodes and edges from the current LCN, but additionally from previous windows, applying a degrading factor the contribution of their edge weights. To build an LCN from a sliding frame of $T$ time windows, the new LCN includes the union of the nodes and edges of the individual LCNs from the current and previous windows, but to calculate the edge weights, we apply a decay factor, $\alpha$, to the weights of edges appearing in windows before the current one. In this way, we apply a multiplier of $\alpha^{x}$ to the edge weights, where $x$ is the number of windows into the past: the current window is $0$ windows into the past, so its edges are multiplied by $\alpha^{0}=1$; the immediate previous window is $1$ window back, so its edge multiplier is $\alpha^{1}$; the one before that uses $\alpha^{2}$, and so on until the farthest window back uses $\alpha^{T-1}$. Generalising from Step 4, the weight $w^{c,t}(e)$ for an edge $e\in E$ between accounts $u$ and $v$ for criterion $c$ at window $t$ and a sliding window $T$ windows wide is given by

In this way, to create a baseline in which the sliding frame is only one window wide, one only need choose $T$=$1$, regardless of the value of $\alpha$. As $\alpha\to 1$, the contributions of previous windows’ are given more consideration.

An approach that aids in the management of data with many features is classification through machine learning. This is an approach that has been used extensively in campaign detection, in which tweets are classified, rather than accounts (e.g., Lee et al., 2013; Chu et al., 2012; Wu et al., 2018). As our intent is to determine whether HCCs detected in datasets are similar to those detecting in ground truth, it is acceptable to rely on one-class classification (i.e., an HCC detected in a dataset is recognised as COORDINATING/positive, or NON-COORDINATING/unknown). The more common binary classification approach was used by Vargas et al. (2020), however our approach has two distinguishing features:

1. 1.

   We rely on one-class classification because we have positive examples of what we regard as COORDINATING from the ground truth, and everything else is regarded as unknown, rather than definitely ‘not coordinating’. A one-class classifier can, for example, suggest a new book from a wide range (such as a library) based on a person’s reading history. In such a circumstance, the classifier designer has access to positive examples (books the reader likes or has previously borrowed) but all other instances (books here) are either positive or negative. When our one-class classifier recognises HCC accounts as positive instances, it provides confidence that the HCC members are coordinating their behaviour in the same manner as the accounts in the ground truth.

2. 2.

   We rely on features from both the HCCs and the HCC members and use the HCC members as the instances for classification, given it is unclear how many members an HCC may have, and accounts that are members of HCCs may have traits in common that are distinct from ‘normal’ accounts. For this reason the feature vectors that our classifier is trained and tested on will comprise features drawn from the individual accounts and their behaviour as well as the behaviour of the HCC of which they are a member. Feature vectors for members of the same HCC will naturally share the feature values drawn from their grouping.

Regarding the construction of the feature vector, at a group level, we consider not just features from the HCC itself, which is a weighted undirected network of accounts, but of the activity network built from the interactions of the HCC members within the corpus. The activity network is a multi-network (i.e., supports multiple edges between nodes) with nodes and edges of different types. The three node types are account nodes, URL nodes, and hashtag nodes. Edges represent interactions and the following types are modelled: hashtag uses, URL uses, mentions, repost/retweets, quotes (cf. comments on a Facebook or Tumblr post), reply, and ‘in conversation’, meaning that one account replied to a post that was transitively connected via replies to an initial post by an account in the corpus. This activity network therefore represents not just the members of the HCC but also their degree of activity.

Although coordinated behaviour in online campaigns is often conducted without automation (Starbird et al., 2019), automation is still commonly present in campaigns, especially in the form of social bots, which aim to present themselves as typical human users (Ferrara et al., 2016; Cresci, 2020). For this reason, the technique presented here is a valid tool for exposing teams of cooperating bot and social bot accounts. We use the Botometer (Davis et al., 2016) service to evaluate selected accounts for bot-like behaviour, which provides a detailed assessment of an account’s botness based on over $1,000$ features in six categories. Each category is assigned a value from $0$, meaning human, to $1$, meaning automated. Although other studies have relied on $0.5$ as a threshold for labelling an account as a bot, there is a a significant overlapping area between humans that act in a very bot-like manner, and bots that are quite human-like, so we adopt the approach of Rizoiu et al. (2018) and regard scores below $0.2$ to be human and scores above $0.6$ to be bots.

How similar are the discovered HCCs to each other and to the rest of the corpus? The HCC detection methods used relied on network information; in contrast we examine content, metadata and temporal information to validate the results. We contrast DS1 and DS2 results with GT and a RANDOM dataset, constructed to match the HCC distributions in DS1 (FSA_V, $\gamma$=$15$). As DS2 consisted entirely of bad actors, and GT consisted entirely of political accounts, it was felt non-HCC accounts from DS1 would be more representative of non-coordinating ‘normal’ accounts.

The IRRs and IMRs for the HCCs in the DS1, DS2, GT and RANDOM datasets is shown in Figure 13. The larger the HCC size, the greater the likelihood of retweeting or mentioning internally, so it is notable that DS2’s largest HCC has IRR and IMR’s of around $0$, though even the smaller HCCs have low ratios. Ratios for the smallest HCCs seem largest, possibly due to low numbers of posts, many of which may be retweets or include a mention, inflating the ratios. The hypothesis that political accounts would retweet and mention themselves frequently is not confirmed by these results, possibly because they are retweeting and mentioning official or party accounts outside the HCCs.

We compared the entropy of features used by DS1 and DS2 HCCs to RANDOM ones (Figure 14). Many of DS1’s small HCCs used only one of a particular feature, resulting in an entropy score of $0$ (Figure 14a). In contrast, DS2’s fewer HCCs have higher entropy values (Figure 14b), likely because they were active for longer (over $365$, not $18$, days) and had more opportunity to use more feature values. The majority of HCCs used few hashtags and URL domains, which is to be expected as the dominating domain is twitter.com, which is embedded in all retweets as links back to the original tweet. Compared to the RANDOM HCCs (Figure 14c), DS1 HCCs had lower variation in all features, while the longer activity period of DS2 resulted in distinctly different entropy distributions. Because DS1 HCC activity appears to have been more deliberate, and perhaps coordinated, it may be that the HCCs were more focused on their topic of conversation (especially when contrasted with RANDOM HCCs). Compared with RANDOM HCCs, DS1 HCCs retweeted fewer accounts, used fewer URLs (though they were from a similar distribution of domains), and many fewer mentions and hashtags. Many non-HCC accounts posted only a single retweet as their contribution to the discussion, and so it may be that a relatively high proportion the RANDOM HCC members only posted a single tweet, causing the distributions observed. As noted previously, the RANDOM HCC members posted $3,147$ tweets compared with DS1 HCCs’ $8,527$ tweets, despite having the same number of members, so DS1 HCC members posted more than $2.7$ times as often. DS1 accounts clearly posted more tweets per individual than the RANDOM accounts, yet their distribution appears similar, and notably different to those of both DS2 and GT (Figure 15).

The sliding frame technique from Section 3.2.9 was applied to DS1 and DS2 to reveal HCCs engaging in coordination consistently in adjacent time windows. The baseline used $T$=$1$ (i.e., a sliding frame a single time window wide) and $\alpha$=$0.0$. For the three other conditions, $T$ was set to $5$ as $\gamma$ increases approximately $5$ times each time, and $\alpha$=$\{0.5,0.7,0.9\}$. In this way, the choice of $\alpha$=$0.9$ would most strongly consider the contribution of LCNs from preceding time windows. Once applied for each time window, the aggregated LCNs were mined for HCCs and then the membership of these were compared in the same manner as in Section 4.3.1 using Jaccard similarity (Equation 2), which, as noted earlier, is stricter about set matching than the Overlap method (Equation 3). Even so, as can be seen in Figure 16, changes introduced by using the decaying sliding frame with different $\alpha$ values were insignificant in all cases, except for DS1 and $\gamma$=$60$. The implication, which is borne out when the exact network sizes (in nodes) are compared in Table 6, is that the previous windows did not add significant numbers of nodes, but instead increased the weight of existing edges, so the HCCs that were detected consisted of the same members working together over time, rather than splitting into subsets. To hide a team’s coordination, one might expect that its members would associate separately in different time window, but that does not appear to have happened significantly in these datasets, except in the shorter time windows in DS1.

The greatest variation in node and edge count occurs in the shorter windows in DS1 ($\gamma$=$\{15,60\}$), probably because of the greater number of accounts active in DS1 (compared to DS2): accounts have more alters to form HCCs with in DS1, which has $20.5$k accounts, whereas choice in DS2 is limited to $1.3$k accounts. The near doubling of accounts in DS1’s HCCs when $\gamma$=$60$ implies accounts co-retweeted often just within a single hour, and then not again (at least not for $T$=$5$ hours). This effect is swamped by the much more active consistent co-retweeting of a smaller set of users when $\alpha$ is increased to $0.7$ and above. Given the membership varies so little in the other conditions, an analysis of how these HCCs form and change over time is required. It is clear, however, that this approach would be best suited to filter-based collections, as they are likely to capture more accounts.

Our final validation method relies on the HCCs in GT as positive examples of coordinating sets of accounts, given it is reasonable to assume that they ought to be coordinating their activities during an election campaign.

Some strategies can involve a combination of actions. Behaviours that contribute to Bullying by dogpiling, for example, include joining conversations started by the target’s posts and mentioning the target repeatedly, within a confined time frame. As DS1 included all replied to tweets, we investigated it inferring links via co-mentions and co-convs with a window size of $10$ minutes, and FSA_V with $\theta$=$0.001$, having maximised the ratio of MEW to HCC size. Of $142$ HCCs discovered, the largest had five accounts and most only had two. Only $32$ had more than ten inferred connections, but five have more than $1,000$. These heavily connected accounts, after deep analysis, were simply very active Twitter users who engaged others in conversation via mentions, which outweighed the more strict co-conv criterion of participants replying into the same conversation reply tree.

A larger window size was considered ($\gamma$=$360$) in case co-conv interactions were more prevalent. FSA_V ($\theta$=$0.01$) exposed little further evidence of co-conv (Figure 17), finding $98$ small HCCs again dominated by co-mentions, not many of which had more than one inferred connection, implying most links were incidental; FSA_V did not filter these out.

This provides an argument for a more sophisticated approach to combining LCN edge weights for analysis, instead of Equation (1), and that FSA_V could be modified to better balance HCC size and edge weight. Furthermore, it is likely that bullying accounts will not just co-mention accounts frequently, but have low diversity in the accounts they co-mention, i.e., they repeatedly co-mention a small set of accounts, and spend a disproportional number of their tweets doing so. A further consideration is that participants in long discussions (reply trees) often include the author of the original tweet that sparked the discussion, and it would be misleading to include their account in results, implying that they bullied themselves. Finally, patterns of behaviour that would clearly qualify as conversations were observed in the datasets that did not fit the strict ‘conversation tree’ model: accounts would mention several collocutors at the start of every tweet, but only potentially reply a tweet of one of them to continue the conversation. Importantly, sometimes the mentioned accounts included in tweets were prominent individuals whose names were included not because they were active participants in the conversation, but because the tweeter wanted to draw their attention to the conversation (regardless of the likelihood that the attempt would succeed; e.g., some tweets included references to prominent and busy politicians who would be unlikely to wade into arbitrary online discussions). These nuances are not explored here.

To study the relationships between HCCs, we create two-level networks starting with the HCC network and then adding nodes representing the elements of evidence linking them, known as reason nodes (e.g., the tweets they co-retweet or the hashtags they use in common). Figure 18 shows the largest component after such expansion was conducted on the HCCs in Figure 17. HCC accounts (circles) share colours and the distribution of the reasons for their connection (diamonds) show which other accounts are uniquely mentioned by an HCC and which are mentioned by more than one HCC. Heavy links between HCC accounts with few adjacent reason vertices imply these accounts are mentioning a small set of other accounts on many occasions.

It is possible to Boost an account rather than just a post. Returning to DS2, we sought HCCs from accounts retweeting the same account (FSA_V, $\gamma$=$15$), and found that the hashtag use revealed further insights (Figure 19). No longer does one HCC dominate the hashtags. Instead clear themes are exhibited by different HCCs, but again, they are not the largest HCCs. The red HCC uses #blacklivesmatter and other Black rights-related hashtags (including #blm, #blacktolive, #blackskinisnotacrime, #policebrutality and #btp¹⁴¹⁴14BTP refers to the British Transport Police, the conduct of which was discussed in accounts of the arrest of a Black man at a London train station in mid-2016, e.g., https://www.theguardian.com/uk-news/2016/jul/28/man-complains-after-police-place-spit-hood-over-head-during-arrest-london-bridge.), while the purple HCC uses pro-Republican ones (#maga and #tcot), and the green HCC is more general. Given the number of tweets these HCCs posted over 2016 (at least $16,849$), it is clear they concentrated their messaging on particular topics, some politically charged. It is arguable that their contributions helped inflame tensions and stoke divisions in socially sensitive topics, not just in the United States, but in the UK as well, and at the very least sought to draw the attention of others.

The green HCC may be acting in a distractor role, as previously mentioned, given their contribution of $72,428$ tweets over the year.

The approach presented can be used to perform analytics similar to the Rapid Retweet Network used by Pacheco et al. (2020a), who used it to expose tight clusters of bot-like accounts, which retweeted the same tweet within $10$ seconds of it appearing. We varied this for the DS1 dataset (due to its small nature) and searched for accounts which retweeted the same tweet within $10$ seconds, regardless of the age of the original tweet. We discovered a tight cluster of accounts, most with relatively high Botometer ratings¹⁵¹⁵15The English “all features” score was used as both our datasets are primarily either English speaking or are predominantly aimed at English speaking audiences. (Davis et al., 2016), shown in Figure 20. The bot ratings were as follows: node 26: $0.787$; node 22: $0.381$; node 2: $0.949$; and node 17: $0.464$. All were high relative to the other accounts in the corpus, most of which had scores well below $0.2$; all four were had scores well above $0.2$, but the scores of two were also well above the ‘bot’ threshold of $0.6$. On further inspection, they appeared to support vocational training and left-wing issues and posted retweets almost exclusively, but the content all related to the election. This finding enhances the bot ratings by making it clear which bots (or bot-like accounts) appear to work together. It also raises further questions regarding bot detection systems, however, as some of the accounts appeared to be genuine human, though abnormally active on Twitter. These accounts appeared to work together to actively disseminate messages aligned with their preferred narrative, though with a very low IRR (just shy of $10\%$) despite most of their activity being retweets ($97.7\%$), so to a certain degree it matters not whether they are automated or genuinely human-driven, but whether they are engaging in astroturfing or other inauthentic behaviour. In this circumstance, they may be genuine agenda-driven users, but they were definitely highly attentive to the same sources. Alternatively, when we consider their bot ratings more closely, it is possible that there is a mixture of account types, with node 26, in particular, acting as an automated ‘cheerleader’ for nodes 22 and 17. Relative timings of their posts (to answer whether node 26 consistently was the second co-retweeter when paired with nodes 22 and 17) could reveal support for this hypothesis.

In Table 9 we present the timings observed for the stages of processing for DS1 and DS2 conducted on a Dell Precision 5520 laptop equipped with an Intel Core i7-7820HQ CPU (2.9GHz), 32Gb RAM, and an NVMe PC300 480Gb SSD, running Windows 10. Parsing raw data is relatively cheap, with DS2’s 1.5m tweets processed in just over a minute, and LCN construction dependent on the degree of activity within the time windows and the number of accounts. DS1’s larger account pool increased the (node) size of the networks generated, and all associated post-processing. The size of DS1 LCNs were an order of magnitude greater than DS2’s (in vertices and edges), resulting in correspondingly increasing execution times for aggregation and HCC extraction.

During the Australian summer of 2019-2020, evidence of inauthentic Twitter behaviour emerged on the hashtag #ArsonEmergency (Weber et al., 2020) over an $18$ day period, including a week prior to and more than a week after a study on the matter (Graham and Keller, 2020) was reported in the online tech and then mainstream media (Stilgherrian, 2020). Analysis of the tweets including the hashtag over the period revealed two clearly polarised retweeting communities, one supporting the narrative that arson was the cause of the bushfires and that eco-activism had prevented forest fuel load management (Supporters) and one that countered the narrative, providing evidence that the fires were mostly started by natural or unintended causes (e.g., lightning and sparks from machinery) and the bushfires’ ferocity was exacerbated by climate change (Opposers). The remaining accounts in the dataset were referred to as Unaffiliated.

Differences between the communities’ interaction and information sharing behaviour was apparent. Supporters interacted with other accounts more by using replies, quotes, and mentions more than Opposers or the Unaffilated, as well as more hashtags and external URLs (i.e., referring to domains other than twitter.com), but they retweeted less. Notably, an analysis of the most shared URLs revealed that Opposers shared articles debunking the narrative exclusively, while Supporters shared a mixture of articles, mostly ones supporting or actively discussing their preferred narrative as well as some conspiratorial content. Unaffiliated accounts shared narrative-focused URLs initially, but in the latter phase of the collection they shared debunking articles $9$ times more often.

Analysis of the co-use of URLs revealed further behavioural differences between the communities (Figure 21). Opposers were more focused that Supporters in the URLs they shared, relying on a small set of debunking articles, also heavily shared by Unaffiliated accounts. Supporters were less tightly clustered around particular articles, and did share some debunking material as well as a variety of narrative-aligned articles.

Co-domain analysis identified not just distinct URLs but distinct URL domains favoured by the different communities. Figure 22 shows two clusters of domains: the red one contains domains from a number of conservative and right wing media organisations, while the blue one contains academic and centre and left wing media organisations. Although Supporters mostly referred to the narrative-supporting domains while the Opposers mostly referred to the debunking domains, it is notable that members of both communities referred heavily the ABC and the Guardian, which both published articles debunking the arson theory, often with reports from local fire fighting and law enforcement organisations. What is lost at this level of analysis is the way in which the articles were discussed when mentioned in tweets, including whether the tweets were agreeing or attacking the article content.

These co-analyses reveal how focused the polarised communities in their information sharing activities, contributing to the argument that the targeted efforts of the Opposer community may have helped influence the broader Unaffiliated community into sharing debunking articles.

A second case study making use of these techniques relates to the search for social bots attempting to influence the online discussion surrounding the Democratic and Republican National Conventions in August 2020, at which the parties formally nominate their candidates for the Presidential Election, later in the year. For a $96$ hour period over each $4$-day convention, tweets were filtered using RAPID, starting with #demconvention and #rnc2020 for the Democratic National Convention (DNC) and the Republican National Convention (RNC), respectively. For the three hours prior to the formal collection, RAPID’s topic tracking feature was enabled, which added hashtags used frequently in the discussions observed, resulting in larger sets of filter terms for each convention:

• •

  DNC: #demconvention, #bidenharris, #bidenharris2020, #khive,
  #signsacrossamerica, #unitedforbiden, and #wewantjoe;

• •

  RNC: #rnc2020, #rncconvention, and #nevertrump.

Despite the disparity in hashtags used (i.e., found to occur sufficiently frequently alongside the seed terms), each dataset ultimately comprised approximately $1.5$ million tweets by over $400$ thousand unique users at each convention. Bots are often used to boost tweets, reaching other accounts that follow them, or by flooding hashtag communities or gaming trending algorithms (Woolley, 2016; Hegelich and Janetzko, 2016; Keller et al., 2019; Graham and Keller, 2020; Graham et al., 2020a). Social bots are specifically designed to mimic genuine human users, hiding the fact they are automated (Ferrara et al., 2016; Grimme et al., 2018). They do this to avoid detection, and in doing so can contribute to astroturfing campaigns, artificially boosting narratives while making them appear as simply popular grass roots movements.

By searching for accounts that co-retweeted within a ten second window of each other and then using a minimum normalised edge weight threshold of $0.1$, several HCCs were identified in each convention (see Figure 23). Analysis of the HCC members using Botometer (Davis et al., 2016) found the majority had a Complete Automation Probability (CAP) ratings above $0.6$ (the relatively high threshold used by Rizoiu et al., 2018, as noted earlier). Further analysis of the HCCs’ content provided some indication of their agendas, and examination of their account age and posting rates enabled categorisation into official accounts (verified by Twitter), unofficial reposters (topic-focused aggregators), and accounts that gave the appearance of typical human users. These ‘normal people’, however, posted at very high average daily rates for years, often at far greater rates than previous automation detection methods have used (e.g., 50 tweets a day, Neudert, 2018).

The largest HCC (the large blue HCC in Figure 23b) consisted of a cluster of potential social bot accounts supporting an official political campaign account, @TrumpWarRoom, responsible for $2,085$ tweets during the Republican Convention. For each pair of members in each HCC, we considered the proportion of time that one account retweeted a tweet before the other, to determine if both accounts were potentially working together (in which case, they would be equally likely to retweet a tweet first), or if one was a ‘cheerleader’ for the other (in which case the cheered account would always retweet first, quickly followed by the other account). We found strong evidence that at least three of the accounts were cheerleaders for @TrumpWarRoom, retweeting the same tweet within ten seconds on $214$, $229$, and $89$ occasions over the four day collection period. These particular accounts had daily tweeting rates of $78.7$, $209.4$ and $147.4$ tweets per day for $0.9$, $8.5$ and $3.6$ years, respectively. Given the age of these accounts, it is clear that they have successfully avoided Twitter’s bot scanning processes for some considerable time.

We also applied co-hashtag analysis (FSA_V, $\theta$=$0.3$, $\gamma$=$10$ seconds) to the two datasets and plotted two-level networks of the resulting HCCs with the hashtags they used (Figure 24). Regardless of the content, a number of structures are immediately apparent. These include:

• •

  clusters that are bound around a few yellow diamond hashtag nodes (e.g., DNC clusters 5, 6 and 8) or lie between hashtags (e.g., DNC clusters 2 and 4);

• •

  fan shapes that consist of a small number of accounts using a wide variety of hashtags (e.g., DNC clusters 1 and 7);

• •

  island clusters that are bound by the hashtags they use but are isolated from the broader community which has ignored the hashtags they are using (e.g., DNC clusters 7 and 8).

The fact that the clusters are coloured according to their HCC in Figure 24 highlights how what FSA_V is extracting what it regards as distinct clusters are, in fact, bound by the topics they are discussing (by the hashtags they are co-using). This indicates that there may be benefit in re-introducing the re-stitching step in FSA (Şen et al., 2016) that FSA_V avoids, or also experimenting further with FSA itself. Using conductance cutting (Brandes et al., 2007) for cluster detection aligned better with the visible clusters, but these clustering may be misleading to varying degrees, as it may combine polarised HCCs, as can be seen on closer inspection below.

Several co-hashtag clusters in Figure 24a provide insight into the nature of parts of the online discussion.

• •

  Cluster 5 is closely centred on two hashtags (#goodyear and #ohio) that relate to then US President Donald Trump’s call for a boycott of Goodyear tires¹⁶¹⁶16The Goodyear factory in Ohio banned clothing with political messaging, including the Trump campaign’s MAGA caps, during the election campaign. Source: https://www.abc.net.au/news/2020-08-20/donald-trump-calls-for-goodyear-boycott-over-alleged-maga-ban/12577372 though it is unclear whether the surrounding accounts are for or against the boycott. Several hashtags linked on the left edge of the cluster indicate that some are against, as they refer to support for the then Democratic candidate Vice President Joe Biden.

• •

  The fan cluster at the top consists of two accounts that are attempting to disseminate their message across America, as each hashtag is a US state code (e.g., #ga for Georgia) or a minority group (e.g., #latinos). These hashtags are all apparently unique, apart from the one highlighted just below cluster 1 surrounded by a blue diamond, which is #blm, linking cluster 1 to cluster 2, and the one to the left, which is another state code (#nc for North Carolina).

• •

  Cluster 2 binds a number of HCCs spanning two relatively disjoint hashtags, one being #vote (below the cluster) and the other being the name of a musician who had recently encouraged his fans to vote.

• •

  Cluster 3 is more diffuse than the others and appears to relate to a discussion of data science and big data in the context of the election campaign.

• •

  Cluster 4 appears to join a number of potentially opposed HCCs, as they refer to #trump2020landslide and #snowflakes as well as #epstein¹⁷¹⁷17Jeffrey Epstein was a billionaire arrested for sex crimes before dying in custody, however he was known to Donald Trump, and therefore this hashtag’s use can be seen as an attack on his political campaign. https://www.forbes.com/sites/lisettevoytko/2020/10/18/spider-book-excerpt-how-trumps-presidency-helped-expose-jeffrey-epstein/ and #trumpvirus (a condemnation of the Trump administration’s handling of the response to the COVID-19 pandemic), the final hashtag which links the cluster into the broader community.

• •

  The island clusters 7 and 8 are focused on groups of particular politicians, which were not picked up by the broader community: Republicans who had pledged to vote for the Democratic candidate and US Congress members known to campaign for social equality, respectively.

The links between the clusters are sometimes deceptive. Already, we observed that some single clusters include polarised HCCs, however it is also possible to see internally (politically) consistent clusters that are linked but also contrary in their views. Cluster 1 in Figure 24a is linked to the left by #nc to another left-leaning cluster (calling for gun control), which itself is linked to the left by #america to another small cluster, which is clearly right-leaning (one of its hashtags is #voteredtosaveamerica). These visualisations may highlight how HCCs can be merged, but care must be taken regarding the conclusions to draw from such an action.

Analysis of the co-hashtag HCCs and their hashtags in Figure 24b offers further examples of these observations and offers new insights. Clusters 1 and 2 are joined by the blue diamond-highlighted hashtag, #blacklivesmatter, but cluster 1 is a detractor group (using #alllivesmatter) while cluster 2 is a supporter group using several Black rights-related hashtags. Cluster 4 discusses riots following Black Lives Matter protests in Kenosha, Wisconsin, however while the two sets of hashtags highlighted at the top of the cluster relate mostly to current events (e.g., #kenosha and #covid19 on the left, and #kenoshariots and #thursdaythoughts, plus #walkaway, which links to a small fan, as it is a pro-Republican statement to avoid conflict), the hashtags at the bottom of the cluster are more clearly right wing or conservative in nature, referring to a relevant media organisation, #kag2020 (Keep America Great, a pro-Trump slogan) and #ccot (Christian Conservative on Twitter). Whereas cluster 4 in Figure 24a includes polarised HCCs, the placement of the hashtag nodes they are linked to offers no guidance on how they might be separated, cluster 4 in Figure 24b indicates that an alternative layout algorithm may aid analysis. Cluster 6 represents a concerted anti-Trump effort with many attacking hashtags, but the isolation of the HCC at the cluster’s centre makes it clear that not many of the others tweeting during the RNC took its lead. Cluster 5 is an effort to draw attention to an instance of police brutality, which also did not gain traction with the broader co-hashtag community.