Measuring changes in polarisation using Singular Value Decomposition of Random Dot Product Graphs

Social and political polarisation is an issue of increasing concern in New Zealand [1], and many other countries [2, 3, 4, 5, 6]. While it was initially thought that New Zealand had avoided the populist takeover seen in countries such as the USA [7], radical changes in the government makeup in both 2020 and 2023 have since called this into question. It is therefore important to develop tools to measure changes in polarisation in societies so that it can be detected and countered as early as possible. In this paper we raise issues with existing polarisation measurement methods, particularly bimodality-based analyses, and propose a new method of measurement which we believe accounts for these issues.

New Zealand was considered to have escaped the development of polarisation which had developed in similar nations, particularly with the 2016 election of Donald Trump in the USA [7]. Analysis of voter polarisation in NZ from 2009-2018 showed little evidence of polarisation in that period [8]. Topic modelling of political party manifestos showed similarities between the manifestos of parties on the left and right [9], suggesting that despite their rivalries the parties are not deeply divided. Tan [10] argues that the lack of bipartisanship between NZ’s left and right indicates that the parties are polarised; however, they found that the perceived distances between left and right parties, and between those parties and voters, were stable over time, suggesting that polarisation is not increasing. Satherley et al. [11] find that there is a high level of stability in the partisanship of NZ voters (i.e. they do not frequently change which party they support) in years up until 2017 and note that this creates a risk of polarisation if voters stay committed to a party that becomes increasingly extreme. In another paper, Satherley et al. [12] find that in analysis of polarisation between the Labour, Green, and National parties, NZ European voters and voters of high socio-economic status exhibit more polarisation in their party preferences than other groups. Stanley et al. [13] find that climate change is not a polarising issue for New Zealand, with the voting public largely accepting its existence.

This history of political stability highlights how irregular the NZ election results in 2020 and 2023 have been, with significant implications for potential polarisation. The 2020 election saw the Labour party win more than 50% of the vote, the first outright majority since the change to Mixed Member Proportional government in 1996, and saw the right-wing ACT minor party increase their vote share from 0.5% to 7.58%. The covid-19 pandemic has subsequently become a focus for disinformation producers, prompting concerns about an increase in polarisation since 2020 [14, 15, 16]. The 2023 election gave further new results; despite the return to the right-wing, both the NZ First and ACT parties were required for the National party to form a coalition, creating the first instance of the National party having to negotiate with two coalition partners of this size. As such, analysis of party-based polarisations based on pre-2020s electoral patterns may no longer be accurate, and developing methods that do not rely on evaluation of political parties is one way of circumventing this issue.

A significant polarisation risk in the current New Zealand landscape is the National party agreeing to support ACT’s “Treaty Principles Bill” through the early stages of the parliamentary process (without guaranteeing it will pass into law). This bill relates to the Treaty Of Waitangi, signed between the British Crown and many Māori chiefs in 1840, which gave the Crown the right to establish a government for its subjects in New Zealand, protected Māori rights to manage their own resources and affairs, and gave Māori the same rights as British citizens [17]. While the Waitangi Tribunal (the permanent commission of inquiry tasked with investigating governmental breaches of the Treaty of Waitangi) has found that Māori did not cede sovereignty by signing the Treaty [18], what this means for New Zealand in constitutional terms is complex and still evolving, especially since the country does not have a single written constitutional document [19]. The Treaty is currently applied to law through the “Treaty Principles”, which are drawn from 50 years of case law [20]. Despite not being codified in parliamentary law, the current Treaty Principles are relatively settled [21]. The Treaty Principles Bill defines an entirely new set of Treaty Principles which contradict and overwrite existing case law, removing references to specific Māori interests. The Waitangi Tribunal has argued that in allowing the Treaty Principles Bill to be introduced to parliament, “the Crown’s [Government’s] actions ‘threaten to disfigure or rupture’ the Crown–Māori relationship and ‘could set back the foundational relationships of Aotearoa New Zealand for decades’” [22]. A rise in general racist sentiment, as well as racist hate crimes and violence, has been documented [23]. As such, we think that questions about whether the polarisation may be happening along an axis other than party affiliation are particularly relevant in the current political atmosphere, since there is a risk that these policies will create a polarisation between Māori and pākehā (descendants of British settlers) which may not match the division of political party affiliations.

Polarisation scholarship in New Zealand is particularly influenced from research in the USA, due to the countries both being English-speaking Western democracies and having a long history of cooperation and political alliance; for example, New Zealand’s membership of the “Five Eyes” surveillance alliance began in 1956 and continues to the present day [24].As such, it is useful to cover some key analyses of polarisation in the USA. A 2018 literature review [2] found that the vast majority of polarisation research in the USA focussed on division between the Republicans and Democrats, at the expense of other possible polarisations such as racial segregation between whites and non-whites. Heatherington [25] famously found that the Republican and Democrat parties have no overlap in their policies or voting patterns, finding a bimodal division by inspection (e.g. no overlap between the ideological groups). His focus on the Republican-Democrat axis leads him to claim that the USA was minimally polarised during the 1950s, since there was no bimodal division; however, he does not comment on other possible axes of polarisation, such as the racial segregation which was legally enforced across the country at the time. We believe that this issue of fixation on the Republican-Democrat axis at the expense of other polarisations applies to a great deal of polarisation research on the USA. Most subsequent research uses significance tests such as Hartigan’s Dip test [26] [27]. When investigating social networks, bimodality is expressed as strong in-group/out-group divisions [28], and is often characterised by hostility between the two groups; it is common to consider this hostility as a key sign of polarisation, in addition to the differences in policy positions between the two poles (the Republican and Democrat parties) [29, 30]. On social media, these hostile divisions usually take the form of ”echo chambers” that focus on one set of political views and exclude all others [31, 32].

A common aspect of definitions of polarisation, first proposed by Esteban & Ray [33], is that the polarisation is maximised when the groups are of equal size, as well as strongly divided. Taking group size into account is intended to help tell polarisation conflicts apart from other major social conflicts, such as conflicts over wealth inequality. However, a consequence of this theoretical approach is that sets a rigid upper limit on how severe a polarisation can become if one of the groups is a small minority. We believe that there are cases where a small group is at high risk of being a target of polarisation, such as Middle Eastern and African refugees entering Europe [27]. Similarly, the extreme racial polarisations of South African apartheid and chattel slavery in the USA both occurred with 80%-20% divisions between the poles (though opposite, with a white majority in the USA and minority in South Africa). Given these issues, we are interested in finding definitions of polarisation, and tools for measuring it, that are not as affected by the size of the groups.

We have some specific concerns about the use of bimodality in detecting polarisation. The most important is that use of hypothesis tests can be used to detect when a distribution is polarised, but after the first significant result it is difficult to robustly show that any further increase in polarisation is also significant. That is to say, that if polarisation is measured at times $t$ and $t+1$ with a significant result from a hypothesis test at both, and $t+1$ has a larger test statistic than $t$, it is difficult to show that this increase itself is significant. Doing so would require finding the standard error of the difference (likely by bootstrapping) and using that to compute and evaluate a test statistic based on the observed value of the difference. Most researchers do not do this.. This means that while studies which use bimodality may detect when polarisation first occurs, we are not confident that they are able to claim that an existing polarisation is getting worse. We are therefore interested in developing measurements of polarisation that do not rely on significance testing in this way. A secondary concern is that this approach can struggle when assessing multi-party democracies that do not have a clear left- and right-wing split [34]. Currently, the main approach to multi-party democracies is to divide all the parties into left- and right-wing blocs and then use these blocs as the two poles measured. However, in the New Zealand context, the NZ First party makes it difficult to clearly reduce the field to left and right blocs; they have formed coalition governments with both the Labour and National parties many times, and are frequently in a position to choose between forming a “left wing” or “right wing” government, so they cannot be removed from the analysis without severely misrepresenting the political field. As such, finding a method of measuring polarisation that can better handle this type of situation would be beneficial to researchers in New Zealand and similar countries.

A common preprocessing step in analyses of polarisation is to use a dimensionality reduction algorithm, such as Principal Component Analysis or Canonical Correspondence Analysis, in order to create a single dimension that can be evaluated for bimodality [35]. These analyses do not draw any other information from the principal components, CCA weights or loadings, such as where the inflection point occurs on a scree plot of the principal components; they are simply used to discover the largest principal component or axis (i.e. the largest dimension). An advantage of this approach is that the issue driving polarisation is generated from the data, rather than researchers presuming what it is and risking choosing incorrectly, such as USA research incorrectly focussing on the Republican-Democrat axis – as we have already discussed. However, it does not entirely mitigate the risk of choosing the wrong dimension, since it presumes that polarisation is happening along the first dimension, and it therefore will still miss cases in which polarisation is happening along one of the other dimensions and has not yet grown severe enough to become the largest dimension. Furthermore, if more than one dimension is required to describe the data (i.e. the elbow of the PCA scree plot occurs at a higher value than 1) then results that find that it is bimodal along the first principal component are not accounting for the fact that the data may not be separated at all along the second, third etc. principal components. When analysing a social network, each principal component reflects a source of connections between individuals; as such, even if the first dimension has a bimodal distribution, fully polarised separation of the network is prevented by connections created in other dimensions. This creates a serious risk of detecting polarisation when it is not actually occurring that we believe is not adequately addressed by many studies which use this method. Due to these issues we are interested in developing methods which could capture polarisation occurring in the network before it becomes severe enough to become the largest dimension, and which do not create false positives by eliminating key dimensions .

In some cases, clustering algorithms are used to measure polarisation instead of bimodality. This usually incorporates the same preprocessing step of using PCA or CCA to reduce the data to a small number of dimensions of interest, such as reducing the data to two dimensions to allow visual inspections of concentrations. However, this has similar issues to bimodality testing, as it is difficult to interpret whether the data is polarised when there are more than two clusters. Methods for detecting polarisation through clustering focus on measuring distances between the clusters. A large distance between the clusters can indicate polarisation between the groups if the clusters also have low internal variation [36]. This means that in theory it is possible for many groups to be polarised against each other; however, we believe there are reasons to restrict the definition of polarisation to only the two group case. In order for all n groups to be the same distance apart (i.e. to dislike all other groups the same amount), there must be at least n-1 dimensions in the space that the group clusters occupy. For example, while two groups A and B could reach polarisation in a 1-dimensional space, the introduction of a third group C would make it impossible for all three groups to be polarised as it would not be possible for A, B and C to all be equidistant from each other. As such, a second dimension of measurement must be added in order to allow equidistant separation between three groups, a third dimension for four groups, etc. Each added dimension increases the total amount of space in between the polarised positions, which creates more possible ways for the groups to be in positions other than equidistance (i.e. more ways for them to be “not polarised”). Since the likelihood of all groups becoming equidistant decreases as dimensions of measurement are added, and since it is impossible for n groups to all be equidistant in a space with fewer than n-1 dimensions, we choose to restrict the definition of “polarisation” to the two group case.

In order to comprehensively address the issue of correct issue selection, Baldassarri and Gelman [37] propose a definition of polarisation as increasing correlation in the ideological space, i.e. that pairs of ideological issues asked about in the American National Election Study become more correlated as a result of polarisation, reducing political pluralism and restricting possible ideological opinions, until maximum correlation is reached and an oppositional binary is created. This polarisation stands in comparison to an integrated, non-polarised society which ”is not a society in which conflict is absent, but rather one in which conflict expresses itself through nonencompassing interests and identities”. Using this definition, they did not find that there was increasing correlation in the ideological field of US-American voters pre-2004. However, subsequent research [38] using the same methods found that polarisation rapidly increased between 2004 and 2016. They noted that the increase in polarisation was strongest in the domains of economics and civil rights issues, rather than in the domain of moral issues that the ”culture war” framing of polarisation may suggest. Similarly, DellaPosta [39] conceptualises polarisation as similar to an oil spill, with the increasing correlation in ideological positions spreading polarisation to previously apolitical members of society. The article analyses how the ”belief network” of US-American politics has changed over time, concluding that the network has developed clusters which have reduced the prevalence of cross-cutting ideological positions; this means that pluralism has decreased and polarisation has increased.

Incorporating post-structuralist political theory allows us to expand on this understanding of polarisation as correlation. Ernesto Laclau and Chantal Mouffe [40] begin from the same understanding of pluralism and polarisation as scholars such as Baldassarri and Gelman, but they develop this concept beyond just the ideological field. They argue that ”In a colonized country, the presence of the dominant power is every day made evident through a variety of contents: differences of dress, of language, of skin colour, of customs […] the colonizer is discursively constructed as the anti-colonized.” (p. 128). This is to say that it is possible for elements of a society’s culture that are not explicitly political to become associated with ideologies and drawn into the correlation, further reducing opportunities for political pluralism. Two poles are constructed that are mutually exclusive and have nothing in common, sustained by segregation in all layers of society (e.g. the South African regime of racial apartheid). Note that this is an extension of the argument we made earlier about three-group polarisations; not only are such polarisations unlikely, but the social dynamics that create polarisation prevent them from occurring, because any pair of groups has a “common enemy” of the remaining group. This “common enemy” prevents the groups from becoming completely separated from each other. Such a society would become “polarised” when the three blocs become collapsed into two fully opposed groups, such as by one of the groups being destroyed or merging with another group. In the case of South African apartheid, this required sorting ethnicities such as Chinese and Japanese into the existing polarisation through declaring them to be “coloured” or “honorary whites” (which did not have exactly the same rights as “white”, but more than “coloured”); this approach preserved the overall two-group polarisation by attaching these ethnicities to the two existing poles rather than allowing them to form a clear third group of their own [41, 42]. Finally, the polarisations that Laclau and Mouffe examine do not occur primarily in the division between political parties, but along fault lines such as race and ethnicity, economic class, the urban-rural divide, and the division between coloniser and colonised. We therefore think that it would be beneficial to develop methods that do not presuppose that political polarisation is happening between the left- and right-wing political blocs, as this may not capture the underlying polarisation correctly.

In this paper we present a novel method of measuring polarisation that follows from both Laclau Mouffe [40], and Baldassarri and Gelman [37], namely using Singular Value Decomposition to determine the embedded dimensionality of social networks. We draw from similar uses of this method in ecology to find the number of niches in an ecological community [43]. Since the embedded dimensionality corresponds to the number of uncorrelated singular values, this corresponds to how much correlation there is in the structure of the network. Using social networks allows us to capture interactions between people that are not explicitly political — being neighbours, sharing a workplace, etc — but which become politicised and segregated during extreme polarisation. As such, we are able to determine whether correlation is increasing not just among possible political positions, but whether it is increasing among social determinants of interaction as well. Our method gives a value that corresponds to the network’s capacity for complexity, and is inversely related to its level of polarisation. Incorporating these additional layers of a society into the analysis should make it easier to detect whether polarisation is occurring.

For our new method of measuring polarisation, we define the mathematics of Random Dot Product Graphs and how to use them to infer the properties of a given network graph that indicate whether it is becoming more or less polarised. We also discuss which method we use for determining the optimal embedded dimensionality of the network graph, since this is the parameter that indicates whether its polarisation is changing. Finally, we add SVD entropy to our analysis to evaluate it as another potential method for measuring the complexity of a network graph.

We apply our computational framework for polarisation to three different data sets: two from Twitter, and one consisting of simulated interaction networks. These have been chosen in order to show that our method works well, and to explore some common beliefs about polarisation in a new way.

We have demonstrated a novel method for measuring polarisation using the embedded dimensionality $\hat{d}$ of random dot-product graphs. This is a reliable and straightforward implementation of the correlation-based approach to polarisation suggested by Baldassarri and Gelman [37], and supported with observations from political scientists Laclau and Mouffe [40]. Our method captured the presence of polarisation in all the scenarios where it was expected and had been found by other researchers, in both simulated data and real social media networks. In particular, the ability to observe the changes in $\hat{d}$ over time in data from the COP conferences provided a useful demonstration of how this method can be applied to longitudinal data. The RDPG approach also allows us to easily see that the process of polarisation is occurring in a network over time, through its embedded dimensionality reducing, rather than relying on a binary test of whether the network was polarised or non-polarised.

Another advantage of the RDPG approach is that it is computationally light; the main bottleneck is the computation of the first singular values of a large matrix, but this is well known in computer science literature and has already been strongly optimised. We found that the SVD was feasible even when used on networks with millions of nodes. Bimodality-based methods typically use SVD or correspondence analysis to determine the dimension they will test for bimodality, so our approach is at least as efficient.

Our approach is also highly interpretable, without forcing the latent ideological distributions into an artificially unidimensional space. Rather than creating a unidimensional space and then interpreting its political meaning (such as pro- and anti-climate, or left- and right-wing), the dimensionality method instead focuses on the number of dimensions rather than what those dimensions are. In high-dimensional spaces, we do not need to know exactly what ideologies the dimensions correspond to; the important part is that they signal that there are connections being made between nodes that would not be possible if the network was polarised.

The SVD entropy of the network did not relate to $\hat{d}$ as closely as we expected, though it did reflect major changes in the networks when they occurred. As such, we think it is best to use $\hat{d}$ of the network to measure its polarisation.

A major limitation of this method which could be improved is that it does not capture affective polarisation very well. Our method functionally considers any interaction between two nodes to be “good”; this means that it is not capable of capturing antagonistic interactions between nodes, and as a result it may overestimate $\hat{d}$ of the network by mistaking brief antagonistic reactions for positive social bonds. There is a great deal of scope for integrating the concept of affective polarisation into our model, through methods such as using signed matrices and classification systems such as sentiment analysis to determine whether interactions in a social network are positive, negative, or neutral before determining $\hat{d}$.

Another possible extension of this method would be to implement a nonparametric two-sample hypothesis test[51], since this would allow a hypothesis test of whether the two networks are significantly different as additional evidence of polarisation having occurred. We believe that being able to observe the embedded dimensionality of the graph alone is useful; however, we understand that sometimes a hypothesis test is demanded, and we believe this would help demonstrate that changes in the embedded dimensionality of the network are significant.

In our tests using stochastic block models, we found an expected result that $\hat{d}$ increases as the probability of edges forming between the groups increases. This is evidence for the common belief about polarisation that connections between the groups decrease their polarisation. We also found an unexpected result that $\hat{d}$ does not straightforwardly increase when one group is much larger than the other, and in fact decreases greatly when one group is 100 times the size of the other. While this is somewhat a property of the size of the network and its sparsity, since the small group was very small and thus unable to make a large number of edges with the big group, we believe that it is an important case to have tested and that closer investigation of this scenario is warranted. We think that allowing the blocks to become sparsely connected without correcting this property is important for modelling the real conditions of social networks at risk of polarisation, such as communities who have recently received a small number of refugees, and models should be carefully designed to ensure that these effects are retained when appropriate.

It would also be productive to experiment more with stochastic block models, since we only tested two possible scenarios. For example, this paper has only explored the two-block case; many instances of online ”echo chambers” have a large number of groups who all hate people different to them, and it would be useful to see what happens to the embedded dimensionality in such cases. Similarly, our testing on group prevalence showed a decrease in embedded dimensionality when one group was 100 times the size of the other, but we did not follow the smaller group’s size all the way to zero so we do not know whether the embedded dimensionality remains low or rebounds to a higher value when one group becomes extremely small.