^affil0^affil0affiliationtext: Institute for Computer Science and Control (SZTAKI)
Eötvös Loránd Research Network (ELKH)
Laboratory on Engineering and Management Intelligence, Research Group of Operations Research and Decision Systems^affil1^affil1affiliationtext: Corvinus University of Budapest (BCE)
Department of Operations Research and Actuarial Sciences^affil2^affil2affiliationtext: Budapest, Hungary

Quantifying the unfairness of the
2018 FIFA World Cup qualification

László Csató Corresponding author. E-mail: [email protected]

Abstract

This paper investigates the fairness of the 2018 FIFA World Cup qualifying competition via Monte Carlo simulations. The qualifying probabilities are calculated for 102 nations, all teams except for African and European countries. A method is proposed to quantify the degree of unfairness. Although the qualifications within four FIFA confederations are constructed fairly, serious differences are found between the continents: for instance, a South American team could have tripled its chances by playing in Asia. Choosing a fixed matchup in the inter-continental play-offs instead of the current random draw can reduce the unfairness of the competition. The move of Australia from the Oceanian to the Asian zone is shown to increase its probability of participating in the 2018 FIFA World Cup by about 65%. Our results provide important insights for the administrators on how to reallocate the qualifying berths.

Keywords: FIFA World Cup; OR in sports; simulation; soccer; tournament design

MSC class: 62F07, 68U20

JEL classification number: C44, C63, Z20

$\mathfrak{Aber}$ $\mathfrak{so}$ $\mathfrak{viel}$ $\mathfrak{ist}$ $\mathfrak{an}$ $\mathfrak{sich}$ $\mathfrak{klar}$ , $\mathfrak{da\ss}$ $\mathfrak{dieser}$ , $\mathfrak{wie}$ $\mathfrak{jeder}$ $\mathfrak{Gegenstand}$ , $\mathfrak{der}$ $\mathfrak{unser}$ $\mathfrak{Begreifungsverm\ddot{o}gen}$ $\mathfrak{nicht}$ $\mathfrak{\ddot{u}bersteigt}$ , $\mathfrak{durch}$ $\mathfrak{einen}$ $\mathfrak{untersuchenden}$ $\mathfrak{Geist}$ $\mathfrak{aufgehellt}$ $\mathfrak{und}$ $\mathfrak{in}$ $\mathfrak{seinem}$ $\mathfrak{inneren}$ $\mathfrak{Zusammenhang}$ $\mathfrak{mehr}$ $\mathfrak{oder}$ $\mathfrak{weniger}$ $\mathfrak{deutlich}$ $\mathfrak{gemacht}$ $\mathfrak{werden}$ $\mathfrak{kann}$ , $\mathfrak{und}$ $\mathfrak{das}$ $\mathfrak{allein}$ $\mathfrak{reicht}$ $\mathfrak{schon}$ $\mathfrak{hin}$ , $\mathfrak{den}$ $\mathfrak{Begriff}$ $\mathfrak{der}$ $\mathfrak{Theorie}$ $\mathfrak{zu}$ $\mathfrak{verwirklichen}$ .¹¹1 “But so much is evident in itself, that this, like every other subject which does not surpass our powers of understanding, may be lighted up, and be made more or less plain in its inner relations by an enquiring mind, and that alone is sufficient to realise the idea of a theory.” (Source: Carl von Clausewitz: On War, Book 2, Chapter 3—Art or Science of War, translated by Colonel James John Graham, London, N. Trübner, 1873. http://clausewitz.com/readings/OnWar1873/TOC.htm)

(Carl von Clausewitz: Vom Kriege)

\justify

1 Introduction

The FIFA World Cup, the most prestigious soccer tournament around the world, is followed by millions of fans. According to Palacios-Huerta, (2014), 5% of all the people who ever lived on the Earth watched the final of the 2010 FIFA World Cup played by the Netherlands and Spain. Qualification to the FIFA World Cup creates widespread media coverage in the competing countries (Frawley and Van den Hoven,, 2015) and brings significant economic benefits (Stone and Rod,, 2016): each participating team has received at least 9.5 million USD in the 2018 FIFA World Cup (FIFA,, 2017). Success in soccer can even help build nations (Depetris-Chauvin et al.,, 2020).

Obviously, every national team cannot play in the FIFA World Cup. Thus there is a World Cup qualification competition, consisting of a series of tournaments organised by the six FIFA confederations. In 2006, Australia left the Oceania Football Confederation (OFC) to join the Asian Football Confederation (AFC). Since then, the country qualified for all FIFA World Cups (2010, 2014, 2018), although it played only twice before (1974, 2006). Is this a mere coincidence, the result of a fruitful team development strategy, or can be partially explained by the change of affiliation? The current paper discusses the third issue by analysing the fairness of the 2018 FIFA World Cup qualification.

Some scientific research has addressed the World Cup qualifiers. Flégl, (2014) applies Data Envelopment Analysis (DEA) to evaluate the performance of national soccer teams during the 2014 FIFA World Cup qualification. Stone and Rod, (2016) aim to assess the degree to which the allocation of qualification berths among the six FIFA confederations reflects the quality of the teams from their specific region, mainly by descriptive statistics. Durán et al., (2017) recommend integer programming to construct alternative schedules for the South American qualifiers that overcome the main drawbacks of the previous approach. Their proposal has been unanimously approved by the South American associations to use in the 2018 FIFA World Cup qualification. Pollard and Armatas, (2017) investigate home advantage in the FIFA World Cup qualification games. Csató, 2020a identifies an incentive incompatibility problem in the European section of the recent FIFA World Cup qualifications.

However, the FIFA World Cup qualification competition has never been studied before with respect to the chances of the participating teams. A possible reason is the complexity of the qualifying system as will be seen later.

Our paper aims to fill this research gap. In particular, the probability of qualification to the 2018 FIFA World Cup is quantified for the 102 nations of the AFC, CONCACAF (Confederation of North, Central American and Caribbean Association Football), CONMEBOL (South American Football Confederation), and OFC to answer three questions: (a) Is the qualification process designed fairly in the sense that it provides a higher chance for a better team both within and between the confederations? (b) Is it possible to improve fairness without reallocating the qualifying berths? (c) How did the move of Australia from the OFC to the AFC in 2006 affect the teams?

The main contributions can be summarised as follows:

1.

First in the academic literature, the paper calculates the qualifying probabilities for the FIFA World Cup based on the Elo ratings of the national teams.
2.

A method is proposed to measure the degree of unfairness. It shows that all the four qualifiers are constructed fairly. On the contrary, substantial differences are found between the confederations.
3.

Using a well-devised fixed matchup in the inter-continental play-offs—a policy applied in the 2010 FIFA World Cup qualification—instead of the current random draw can reduce unfairness by 7.5%.
4.

Australia has increased its probability of playing in the 2018 FIFA World Cup by 65% as a result of leaving the OFC and joining the AFC. The move has been detrimental to any AFC nation, while it has favoured all other countries, especially New Zealand. The change has reduced unfairness by almost 20%, which might explain why the move of Australia was approved.

Our approach can be applied in any sports where teams contest in parallel tournaments for the same prize, hence the natural issue of equal treatment of equals emerges. While these designs have recently been analysed with respect to incentive incompatibility (Vong,, 2017; Dagaev and Sonin,, 2018; Csató, 2020b, ), the numerical investigation of fairness is currently limited to the qualification for the UEFA Euro 2020 (Csató, 2021b, , Chapter 6.4).

2 Related literature

The paper contributes to three different fields of research: fairness in sports, analysis of FIFA competitions and rules, and simulation of tournament designs.

Fairness in sports is a widely discussed topic in the literature. From the perspective of tournament design, there are two crucial axioms: (1) stronger players should be preferred to weaker players; and (2) equal players should be treated equally. Otherwise, an incentive might exist to manipulate the tournament. Groh et al., (2012) check which seedings in elimination tournaments satisfy the two properties. Arlegi and Dimitrov, (2020) apply these requirements to different kinds of knockout contests and characterise the appropriate structures. Csató, 2021b (, Chapter 6.4) shows the unfairness of the qualification for the 2020 UEFA European Championship with important lessons for sports management (Haugen and Krumer,, 2021). Both theoretical (Krumer et al.,, 2017, 2020; Laica et al.,, 2021; Sahm,, 2019) and empirical (Krumer and Lechner,, 2017) investigations reveal that the ex-ante winning probabilities in round-robin tournaments with three and four symmetric players may depend on the schedule, which can lead to severe problems in the 2026 FIFA World Cup (Guyon,, 2020). Soccer penalty shootouts seem to be advantageous for the first shooter (Apesteguia and Palacios-Huerta,, 2010; Palacios-Huerta,, 2014; Vandebroek et al.,, 2018) but this bias can be mitigated by a carefully devised mechanism (Anbarcı et al.,, 2021; Brams and Ismail,, 2018; Csató, 2021a, ; Csató and Petróczy,, 2022; Lambers and Spieksma,, 2021; Palacios-Huerta,, 2012). The knockout bracket of the 2016 UEFA European Championship has created imbalances among the six round-robin groups of four teams each (Guyon,, 2018). Vaziri et al., (2018) state some fairness properties of sports ranking methods.

In contrast to the World Cup qualifiers, several attempts have been made to forecast the FIFA World Cup final tournament games. Dyte and Clarke, (2000) treat the goals scored by the teams as independent Poisson variables to simulate the 1998 FIFA World Cup. Deutsch, (2011) aims to judge the impact of the draw in the 2010 World Cup, as well as to look back and identify surprises, disappointments, and upsets. Groll et al., (2015) fit and examine two models to forecast the 2014 FIFA World Cup. O’Leary, (2017) finds that the Yahoo crowd was statistically significantly better at predicting the outcomes of matches in the 2014 World Cup compared to the experts and was similar in performance to established betting odds. Groll et al., 2019a propose a novel hybrid modelling approach by combining random forests with Poisson ranking methods to predict the 2018 FIFA World Cup. This technique has also been used to forecast the 2019 FIFA Women’s World Cup (Groll et al., 2019b, ) and the 2020 UEFA European Championship (Groll et al.,, 2021).

Further aspects of the FIFA World Cup have also been researched extensively. Jones, (1990) and Rathgeber and Rathgeber, (2007) discuss the consequences of the unevenly distributed draw procedures for the 1990 and 2006 FIFA World Cups, respectively. Scarf and Yusof, (2011) reveal the effect of seeding policy and other design changes on the progression of competitors in the World Cup final tournament. Guyon, (2015) collects some flaws and criticisms of the World Cup seeding system. Laliena and López, (2019) and Cea et al., (2020) provide a detailed analysis of group assignment in the FIFA World Cup.

Finally, since historical data usually do not make it possible to calculate the majority of tournament metrics such as qualifying probabilities, it is necessary to use computer simulations for this purpose, especially for evaluating new designs (Scarf et al.,, 2009). Any simulation model should be based on a prediction model for individual ties. According to Lasek et al., (2013), the best performing algorithm of ranking systems in soccer with respect to accuracy is a version of the famous Elo rating. Baker and McHale, (2018) provide time-varying ratings for international soccer teams. Van Eetvelde and Ley, (2019) overview the most common ranking methods in soccer. Ley et al., (2019) build a ranking reflecting the teams’ current strengths and illustrate its usefulness by examples where the existing rankings fail to provide enough information or lead to peculiar results. Corona et al., (2019) propose a Bayesian approach to take into account the uncertainty of parameter estimates in the underlying match-level forecasting model.

3 The 2018 FIFA World Cup qualification

The FIFA World Cup qualification is a series of tournaments to determine the participants of the FIFA World Cup. Since 1998, the final competition contains 32 teams such that the host nation(s) receive(s) a guaranteed slot. The number of qualifying berths for the continents is fixed from 2006 to 2022 as follows:

•

AFC (Asian Football Confederation): 4.5;
•

CAF (Confederation of African Football): 5;
•

CONCACAF (Confederation of North, Central American and Caribbean Association Football): 3.5;
•

CONMEBOL (South American Football Confederation): 4.5;
•

OFC (Oceania Football Confederation): 0.5;
•

UEFA (Union of European Football Associations): 13.

The six confederations organise their own contests. The 0.5 slots represent a place in the inter-continental play-offs, which is the only interaction between the qualifying tournaments of different geographical zones.

The qualifications of all confederations are played in rounds. Each round is designed either in a knockout format (where two teams play two-legged home-away matches) or in a round-robin format (where more than two teams play in a single or home-away group against every other team of the group). The rounds are often seeded, that is, the participating countries are divided into the same number of pots as the number of teams per group (meaning two pots in the knockout format) and one team from each pot goes to a given group. The traditional seeding is based on an exogenously given ranking—usually the FIFA World Ranking at a specific date—such that, if a pot contains $k$ teams, the best $k$ teams are in the first pot, the next $k$ are in the second pot, and so on.

Our paper focuses on four qualifications, the AFC, the CONCACAF, the CONMEBOL, and the OFC because (1) contrary to the CAF and UEFA competitions, they are connected to each other; (2) the largest and most successful nation of the OFC, Australia, switched to the AFC in 2006.

The 2018 FIFA World Cup qualification (AFC) contained $46$ nations and four rounds. The starting access list was determined by the FIFA World Ranking of January 2015.

•

First round
Format: knockout
Competitors: the $12$ lowest-ranked teams ( $35$ – $46$ )
Seeding: traditional; based on the FIFA World Ranking of January 2015
•

Second round
Format: home-away round-robin, 8 groups of five teams each
Competitors: the $34$ highest-ranked teams ( $1$ – $34$ ) + the six winners from the first round
Seeding: traditional; based on the FIFA World Ranking of April 2015²²2 Since the seeding order differed from the ranking in the AFC entrant list, three winners in the first round (India, Timor-Leste, Bhutan) were not seeded in the weakest pot 5.
•

Third round
Format: home-away round-robin, 2 groups of six teams each
Competitors: the eight group winners and the four best runners-up in the second round³³3 Group F in the second round consisted of only four teams because Indonesia was disqualified by FIFA. Therefore, the matches played against the fifth-placed team were disregarded in the comparison of the runners-up.
Seeding: traditional; based on the FIFA World Ranking of April 2016
The two group winners and the two runners-up qualified to the 2018 FIFA World Cup.
•

Fourth round
Format: knockout
Competitors: the third-placed teams from the groups in the third round
Seeding: redundant
The winner advanced to the inter-confederation play-offs.

The 2018 FIFA World Cup qualification (CONCACAF) contained $35$ nations and five rounds. The access list was determined by the FIFA World Ranking of August 2014.

•

First round
Format: knockout
Competitors: the $14$ lowest-ranked teams ( $22$ – $35$ )
Seeding: traditional; based on the FIFA World Ranking of August 2014
•

Second round
Format: knockout
Competitors: the teams ranked $9$ – $21$ in the access list + the seven winners from the first round
Seeding: the seven teams of pot 5 (ranked $9$ – $15$ ) were drawn against the teams of pot 6 (the winners from the first round) and the three teams of pot 3 (ranked $16$ – $18$ ) were drawn against the three teams of pot 4 (ranked $19$ – $21$ ); based on the FIFA World Ranking of August 2014
•

Third round
Format: knockout
Competitors: the teams ranked $7$ – $8$ in the access list + the $10$ winners from the second round
Seeding: traditional; based on the FIFA World Ranking of August 2014
•

Fourth round
Format: home-away round-robin, 3 groups of four teams each
Competitors: the teams ranked $1$ – $6$ in the access list + the six winners from the third round
Seeding: pot 1 (teams ranked $1$ – $3$ ), pot 2 (teams ranked $4$ – $6$ ), pot 3 (the winners from the third round) such that each group contained a team from pot 1, a team from pot 2, and two teams from pot 3; based on the FIFA World Ranking of August 2014
•

Fifth round
Format: home-away round-robin, one group of six teams
Competitors: the group winners and the runners-up in the fourth round
Seeding: redundant
The top three teams qualified to the 2018 FIFA World Cup and the fourth-placed team advanced to the inter-confederation play-offs.

The CONCACAF competition can be criticised for the great role attributed to the FIFA World Ranking. Perhaps it is not only a coincidence that this confederation has planned to fundamentally restructure its qualifying tournament for the 2022 FIFA World Cup (CONCACAF,, 2019).

The 2018 FIFA World Cup qualification (CONMEBOL) contained $10$ nations, which contested in a home-away round-robin tournament (Durán et al.,, 2017). The top four teams qualified to the 2018 FIFA World Cup, and the fifth-placed team advanced to the inter-confederation play-offs.

The 2018 FIFA World Cup qualification (OFC) contained $11$ nations and four rounds.

•

First round
Format: single round-robin, one group organised in a country (Tonga was chosen later)
Competitors: the four lowest-ranked teams ( $8$ – $11$ ), based on FIFA World Ranking and sporting reasons
Seeding: redundant
•

Second round
Format: single round-robin, 2 groups of four teams each, all matches played in one country
Competitors: the seven strongest teams ( $1$ – $7$ ) + the group winner in the first round
Seeding: traditional; based on the FIFA World Ranking of July 2015
•

Third round
Format: home-away round-robin, 2 groups of three teams each
Competitors: the top three teams from each group in the second round
Seeding: pot 1 (2016 OFC Nations Cup finalists), pot 2 (2016 OFC Nations Cup semifinalists), pot 3 (third-placed teams in the second round) such that each group contained one team from pots $1$ – $3$ each⁴⁴4 The group stage of the 2016 OFC Nations Cup served as the second round of the 2018 FIFA World Cup qualification (OFC). Any group winner was matched with the runner-up of the other group in the semifinals of the 2016 OFC Nations Cup.
•

Fourth round
Format: knockout
Competitors: the group winners in the third round
Seeding: redundant
The winner advanced to the inter-confederation play-offs.

Consequently, the inter-confederation play-offs were contested by four teams from the four confederations (AFC, CONCACAF, CONMEBOL, OFC), and were played in a knockout format. The four nations were drawn randomly into two pairs without seeding. The two winners qualified to the 2018 FIFA World Cup. The inter-confederation play-offs of the 2006 FIFA World Cup and the 2014 FIFA World Cup qualification were also drawn randomly. This policy will be followed in the 2022 FIFA World Cup, too. However, FIFA fixed the ties in the inter-continental play-offs of the 2010 FIFA World Cup qualification as AFC vs. OFC and CONCACAF vs. CONMEBOL to pair teams being in closer time zones.⁵⁵5 Similarly, FIFA matched a randomly drawn UEFA runner-up with the AFC team, and two nations from CONMEBOL and OFC in the two inter-continental play-offs of the 2002 FIFA World Cup qualification.

4 Methodology and implementation

This section presents how the outcome of a qualifying match is determined, the simulation is carried out, and the unfairness of a qualification tournament is measured.

4.1 The simulation model

In order to quantify a particular tournament metric of the FIFA World Cup qualification, it is necessary to follow a simulation technique because historical data are limited: the national teams do not play many matches and the outcome of a qualification is only a single realisation of several random variables. Such a model should be based on predicting the result of individual games. For this purpose, the strengths of the teams are measured by the World Football Elo Ratings, available at the website eloratings.net. Elo-inspired methods are usually good in forecasting (Lasek et al.,, 2013), and have been widely used in academic research (Hvattum and Arntzen,, 2010; Lasek et al.,, 2016; Cea et al.,, 2020).

Elo ratings depend on the results of previous matches but the same result is worth more when the opponent is stronger. Furthermore, playing new games decreases the weight of earlier matches. Since there is no official Elo rating for national teams, this approach can be implemented in various ways. For instance, while the official FIFA World Ranking adopted the Elo method of calculation after the 2018 FIFA World Cup (FIFA, 2018a, ; FIFA, 2018b, ), it does not contain any adjustment for home or away games. However, home advantage has been presented to be a crucial factor in international soccer, even though its influence appears to be narrowing (Baker and McHale,, 2018). The World Football Elo Ratings takes into account some soccer-specific parameters such as the margin of victory, home advantage, and the tournament where the match was played.

In the 2018 FIFA World Cup qualification, three types of matches were played: group matches in a home-away format, single group matches in a randomly chosen country that is assumed to be a neutral field (only in the first and the second rounds in the OFC zone), and home-away knockout matches. The win expectancy of team $i$ against team $j$ can be directly obtained from the formula of Elo rating according to the system of World Football Elo Ratings (see http://eloratings.net/about):

W_{ij}=\frac{1}{1+10^{-(E_{i}-E_{j})/400}},

(1)

where $E_{i}$ and $E_{j}$ are the Elo ratings of the two teams, respectively, such that the rating of the home team is increased by $100$ .

In knockout clashes, the teams focus primarily on advancing to the next round rather than winning one match. Therefore, we have followed the solution of the ClubElo rating (see http://clubelo.com/System), namely, these two-legged matches are considered as one long match with a corresponding increase in the difference between the strengths of the teams. The probability that team $i$ advances against team $j$ is:

W_{ij}^{(KO)}=\frac{1}{1+10^{-\sqrt{2}(E_{i}-E_{j})/400}}.

(2)

In the case of group matches, the number of goals scored by each team in each match should also be modelled because drawn matches are common in football, and ties between teams with the same score are decided by goal difference. For this purpose, the number of goals is assumed to follow a Poisson distribution, which is a standard solution in the literature, see, e.g. Chater et al., (2021) or Dagaev and Rudyak, (2019). In particular, team $i$ scores $k$ goals against team $j$ with the probability

P_{ij}(k)=\frac{\left(\lambda_{ij}^{(f)}\right)^{k}\exp\left(-\lambda_{ij}^{(f)}\right)}{k!},

(3)

where the parameter $\lambda_{ij}^{(f)}$ is the expected number of goals scored by team $i$ against team $j$ if the match is played on field $f$ (home: $f=h$ ; away: $f=a$ ; neutral $f=n$ ).

The value of parameter $\lambda_{ij}^{(f)}$ in (3) is modelled as a quartic polynomial of $W_{ij}$ (see (1)) estimated using a least squares regression with a regime change at $W_{ij}=0.9$ (Football rankings,, 2020). On the basis of more than 29 thousand home-away matches played by national football teams, the average expected number of goals for the home team $i$ is

\lambda_{ij}^{(h)}=\left\{\begin{array}[]{ll}-5.42301\cdot W_{ij}^{4}+15.49728\cdot W_{ij}^{3}\\ -12.6499\cdot W_{ij}^{2}+5.36198\cdot W_{ij}+0.22862&\textrm{if }W_{ij}\leq 0.9\\ \\ 231098.16153\cdot(W_{ij}-0.9)^{4}-30953.10199\cdot(W_{ij}-0.9)^{3}&\\ +1347.51495\cdot(W_{ij}-0.9)^{2}-1.63074\cdot(W_{ij}-0.9)+2.54747&\textrm{if }W_{ij}>0.9\end{array}\right.

(4)

with $R^{2}=0.984$ , whereas the average number of goals for the away team $j$ equals

\lambda_{ij}^{(a)}=\left\{\begin{array}[]{ll}90173.57949\cdot(W_{ij}-0.1)^{4}+10064.38612\cdot(W_{ij}-0.1)^{3}\\ +218.6628\cdot(W_{ij}-0.1)^{2}-11.06198\cdot(W_{ij}-0.1)+2.28291&\textrm{if }W_{ij}<0.1\\ \\ -1.25010\cdot W_{ij}^{4}-1.99984\cdot W_{ij}^{3}&\\ +6.54946\cdot W_{ij}^{2}-5.83979\cdot W_{ij}+2.80352&\textrm{if }W_{ij}\geq 0.1\end{array}\right.

(5)

with $R^{2}=0.955$ .

In the OFC zone, some matches are played on neutral ground when, based on almost 10 thousand matches, the average number of goals for team $i$ is

\lambda_{ij}^{(n)}=\left\{\begin{array}[]{ll}3.90388\cdot W_{ij}^{4}-0.58486\cdot W_{ij}^{3}\\ -2.98315\cdot W_{ij}^{2}+3.13160\cdot W_{ij}+0.33193&\textrm{if }W_{ij}\leq 0.9\\ \\ 308097.45501\cdot(W_{ij}-0.9)^{4}-42803.04696\cdot(W_{ij}-0.9)^{3}&\\ +2116.35304\cdot(W_{ij}-0.9)^{2}-9.61869\cdot(W_{ij}-0.9)+2.86899&\textrm{if }W_{ij}>0.9,\end{array}\right.

(6)

with $R^{2}=0.976$ .

The ranking of the teams in a group is determined as follows: (a) greatest number of points obtained in all group matches; (b) goal difference in all group matches; (c) greatest number of goals scored in all group matches; (d) drawing of lots.

The Elo ratings are dynamic but we have fixed them for the sake of simplicity. In each of the four confederations, the ratings of all teams on the day before the first match of the relevant qualification tournament are considered. Four tables in the Appendix show the corresponding measures of strength: Table A.1 for the $35$ CONCACAF teams; Table A.2 for the $46$ AFC teams; Table A.3 for the $10$ CONMEBOL teams; and Table A.4 for the $11$ OFC teams.

Any theoretical model is only as good as its assumptions. Therefore, it is worth summarising the main limitations:

•

The strength of the teams is exogenously given and fixed during the whole qualification process.
•

Home advantage does not differ between the confederations despite the findings of Pollard and Armatas, (2017). However, the influence of the corresponding parameter is minimal since all matches are played both home and away except for Oceania, where some games are hosted by a randomly drawn country.
•

The schedule of the matches is not accounted for.
•

The efforts of the teams do not change even if they have already qualified or have been eliminated.

These shortcomings mean that the numerical results are primarily for comparative purposes.

4.2 Implementation

Our computer code closely follows the rules of the qualification process described in the previous section. Pot assignment and seeding are based on the Elo rating of the teams in each case since the strengths of the teams are given by this measure instead of the FIFA ranking, which was a rather bad predictor before the 2018 reform (Lasek et al.,, 2013; Cea et al.,, 2020). Although the official AFC qualification updated the ranking of the teams before the seeding in each round—thus the results of matches played already during the qualification may have affected the subsequent rounds—, that complication is disregarded in our simulations. Using the Elo rating for seeding is also necessary to guarantee the consistency of the simulations since the proposed fairness measure will be based on the Elo ratings. Therefore, every instance of unfairness will be intrinsic to the tournament design.

Finally, the move of Australia from the OFC to the AFC will also be evaluated. Accordingly, an alternative design of the FIFA World Cup qualification should be chosen with Australia being in the OFC instead of the AFC. Since then there are only $45$ countries in Asia, a straightforward format would be to organise the first knockout round with the 10 lowest-ranked teams ( $36$ – $45$ ), while the second round is contested by the $35$ highest-ranked teams ( $1$ – $35$ ) plus the five winners from the first round. Together with Australia, the OFC qualification contains $12$ teams. Fortunately, the design of the 2006 FIFA World Cup qualification (OFC) can be adopted without any changes:

•

First round
Format: single round-robin, 2 groups of five teams each, held in one country
Competitors: the $10$ lowest-ranked teams, that is, all nations except for Australia and New Zealand
Seeding: traditional⁶⁶6 This is only a (reasonable) conjecture as we have not found the official regulation.
•

Second round
Format: single round-robin, one group of six teams, held in one country
Competitors: the two highest-ranked teams (Australia, New Zealand) + the group winners and the runners-up in the first round
Seeding: redundant
•

Third round
Format: knockout
Competitors: the group winner and the runner-up in the second round
Seeding: redundant
The winner advanced to the inter-confederation play-offs.

The simulation has been carried out with 2.5 million independent runs in each scenario (e.g. when Australia plays in the OFC zone). The probability of qualification is provided by the mean of these iterations, for example, it equals 60% if a national team qualifies in 1.5 million runs. A further increase does not reduce statistical errors considerably and would be a futile exercise anyway in view of the warnings above.

4.3 A reasonable measure of unfairness

The degree of unfairness can be quantified by ranking the teams according to their Elo rating and summing the differences of qualifying probabilities that do not fall into line with this ranking. However, in order to ensure the comparability of different tournaments, it is worth guaranteeing independence of the number of competitors $n$ and the number of qualifying slots $q$ .⁷⁷7 We are grateful to Dries Goossens for this remark. Since the worst case with respect to fairness is when the $q$ lowest-ranked teams qualify with probability one, the crude sum is divided by its theoretical maximum of $(n-q)q$ to get a normalised measure between zero and one.

Definition 1.

Measure of unfairness:

\textit{UF}=\sum_{E_{i}\geq E_{j}}\frac{\max\{0;p(j)-p(i)\}}{(n-q)q},

(7)

where $E_{i}$ and $p(i)$ are the Elo rating and the probability of qualification for team $i$ , respectively, $n$ is the number of competitors and $q$ is the number of qualifying slots.

Formula (7) only considers the ordinal strength of the teams because prescribing how the differences in Elo rating should be converted into differences in qualifying probabilities would require further assumptions, which seem to be challenging to justify.

5 Results

The three main research questions, presented in the Introduction, will be discussed separately.

5.1 Quantifying unfairness

Figure 1: The probability of qualification for the 2018 FIFA World Cup

Figure 1 shows the probability of qualification for the 2018 FIFA World Cup as the function of Elo ratings. Unsurprisingly, the simple round-robin format of the CONMEBOL qualification guarantees that this tournament metric depends monotonically on the strength of the teams. The structure of the OFC qualification does not necessarily satisfy the fairness condition but it still holds because only the four weakest teams should play in the first round and the seeding is based on the strengths of the teams. Similarly, the AFC and CONCACAF qualifications are essentially conforming to the principle of giving higher chances for better teams, too.

Table 1: The level of unfairness within the confederations

Confederation	Value of UF
AFC	$6.23\times 10^{-7}$
CONCACAF	$4.18\times 10^{-6}$
CONMEBOL	0
OFC	0

As Table 1 reveals, the qualification tournaments of all confederations are constructed fairly (the slot in the play-offs is counted as half). The negligible numbers for AFC and CONCACAF are only due to the stochastic nature of the simulation, which leads to volatile qualifying probabilities for weak teams. A team with a lower Elo rating never has a higher chance to qualify than a team with a higher Elo rating if either of these probabilities is above 0.002%. On the other hand, some teams have the same Elo rating (see Tables A.1 and A.2 in the Appendix), and these cases naturally increase UF according to Definition 1.

Unfairness has another dimension, that is, between the confederations. According to Figure 1, the South American qualifiers seem to be the toughest. In order to investigate this issue, Peru (the $7$ th strongest team in CONMEBOL, Elo: 1807) has been exchanged sequentially with the strongest teams in the other three confederations: Japan (AFC, Elo: 1757), Mexico (CONCACAF, Elo: 1893), and New Zealand (OFC, Elo: 1516). These countries are highlighted in Figure 1.

Table 2: Qualifying probabilities when Peru is moved to another confederation

Team	Original	Peru plays in
	confederation	AFC	CONCACAF	CONMEBOL	OFC
Japan	AFC	0.117	0.793	0.794	0.785
Mexico	CONCACAF	0.968	0.508	0.968	0.966
Peru	CONMEBOL	0.887	0.877	0.235	0.455
New Zealand	OFC	0.185	0.183	0.185	$5.67\times 10^{-5}$

Table 2 reports the probabilities of qualification for the four nations if Peru would contest in various confederations. The numbers in the diagonal show that any team is the worst off when playing in the CONMEBOL qualifiers but the chance of Peru to participate in the 2018 FIFA World Cup would more than triple by playing in the AFC or CONCACAF zone. Being a member of the OFC would be less beneficial for Peru due to the lack of a direct qualification slot. Its effect can be seen to some extent in the qualifying probabilities of Japan and Mexico: since Peru would qualify with more than 97% probability from the OFC qualifiers to the inter-confederation play-offs, the two teams would have a larger probability to face Peru there, which would reduce their chances to advance to the World Cup finals.

5.2 A potential improvement of fairness

The straightforward solution to handle unfairness between the confederations would be to reallocate the slots available for them, especially because the current allocation system lacks any statistical validation, does not ensure the qualification of the best teams in the world, and does not reflect the number of teams per federation (Stone and Rod,, 2016). The whole process is far from being transparent and is mainly determined by political, cultural, and historical factors. Consequently, operations research has a limited role to influence the allocation of World Cup slots between the FIFA confederations.

However, the matching in the two inter-confederation play-offs is probably a variable to be chosen freely by the FIFA executives who are responsible for the tournament design, as illustrated by the two policies used recently. We have considered three possibilities:

•

Random draw for the play-offs: the four participants from the confederations AFC, CONCACAF, CONMEBOL, and OFC are drawn randomly into two pairs;
•

Fair draw for the play-offs: the four participants are paired such that AFC vs. CONCACAF and CONMEBOL vs. OFC;
•

Close draw for the play-offs: the four participants are paired such that AFC vs. OFC and CONCACAF vs. CONMEBOL.

The random draw has been used in the 2006 FIFA World Cup qualification, as well as since the 2014 FIFA World Cup qualifiers. The close draw has been used in the 2010 FIFA World Cup qualification competition: it matches nations from closer time zones, which allows for better kick-off times. Therefore, it can be optimal for the players, as well as may maximise gate revenue and the value of television rights. Finally, the fair draw is inspired by Figure 1 and Table 2 since the CONMEBOL team is usually the strongest and the OFC team is usually the weakest in the play-offs.

Table 3: Unfairness and the draw for the play-offs

a The overall level of unfairness

Draw policy for the play-offs
Random	Fair	Close
0.01063	0.00985	0.01308

b The qualifying probabilities of certain teams

Team	Confederation	Elo rating	Draw policy for the play-offs
Team	Confederation	Elo rating	Random	Fair	Close
Australia	AFC	1727	0.708	0.744	0.781
Japan	AFC	1757	0.744	0.778	0.810
Mexico	CONCACAF	1893	0.968	0.975	0.958
Peru	CONMEBOL	1807	0.235	0.241	0.205
New Zealand	OFC	1516	0.184	0.041	0.179

The draws for the play-offs are compared in Table 3, the measure of unfairness (Definition 1) is presented in Table 3.a, while Table 3.b provides the probability of qualification for some countries. Again, there are some countries with the same Elo rating: Guatemala, Honduras (1539); Qatar, Trinidad and Tobago (1525); Jamaica, Oman (1522); Canada, El Salvador (1507); Haiti, Kuwait, North Korea (1446); Malaysia; Philippines (1225); India, Maldives (1068). These cases naturally increase the value of UF, mostly when the national teams play in different confederations.

Intuitively, the fair draw is the least unfair. The close draw mostly favours AFC and OFC, however, it is detrimental to the members of the CONCACAF and CONMEBOL, implying the most severe unfairness.

Figure 2: The effect of improving the fairness of the 2018 FIFA World Cup qualification
via modifying the draw for inter-confederation play-offs

The effect of a fair draw is detailed in Figure 2 for the teams with at least one percentage point change in the probability of qualification to the 2018 FIFA World Cup. Compared to the current random design, all Asian and South American countries would be better off and the strongest CONCACAF countries are also preferred. On the other hand, all nations of the OFC, in particular, the dominating New Zealand, would lose substantially from this reform. The gains are distributed more equally because there is no such a prominent team in the other zones. Some weaker CONCACAF members are worse off due to the impossibility of playing against New Zealand in the inter-confederation play-offs.

5.3 Counterfactual: was it favourable for Australia to join the AFC?

FIFA president Sepp Blatter had promised a full slot to the OFC as part of his re-election campaign in November 2002 but the suggestion was reconsidered in June 2003 (ABC,, 2003). Subsequently, the largest and most successful nation of the OFC, Australia, left to join the AFC in 2006. It raises the interesting question of how this move has affected the 2018 FIFA World Cup qualification.

First, the unfairness measure UF would be $0.013$ with Australia playing in the OFC, which corresponds to an increase of more than 20% compared to the current situation. The action of Australia has contributed to the fairness of the 2018 FIFA World Cup qualification. Furthermore, the magnitude of the improvement is higher than the gain from the proposed novel draw for the inter-confederation play-offs. This reduction in unfairness likely explains why the move of Australia was approved.

Figure 3: The effect of Australia being a member of the AFC
on the 2018 FIFA World Cup qualification

Second, the probabilities of qualification are computed if Australia would have remained in the OFC. Figure 3 plots the effects for the national teams facing a change of at least one percentage point. Notably, Australia has increased the probability of participating in the 2018 FIFA World Cup from 43% to 70% by leaving the OFC for the AFC. The move has also been strongly favourable for New Zealand, which is now the strongest OFC team and has more than 70% chance to grab the slot guaranteed in the play-offs for Oceania. Every CONCACAF and CONMEBOL member has been better off due to the reduction in the expected strength of the countries contesting in the play-offs. However, all original AFC nations have lost with the entrance of Australia, especially those teams that are only marginally weaker than Australia.

6 Conclusions

We have analysed four series of qualification tournaments for the 2018 FIFA World Cup via Monte Carlo simulations. Their design does not suffer from serious problems. However, there are substantial differences between the chances of nations playing in different continents: Peru would have tripled its probability of qualification by competing in the AFC or CONCACAF zone, whereas New Zealand would have lost any prospect of participation by being a member of the CONMEBOL. Australia is found to have greatly benefited from leaving the OFC for the AFC in 2006.

In addition, probably first in the literature, a measure of unfairness has been proposed to quantify to which extent weaker teams are preferred over stronger teams by a tournament design. A simple modification in the format of the inter-confederation play-offs can reduce this metric by about 7.5%, hence FIFA has a good reason to consider the novel policy.

Hopefully, this paper will become only a pioneering attempt of academic researchers to study the qualification to the soccer World Cup. There remains huge scope for improving the simulation model, especially concerning the prediction of individual matches. Nonetheless, even these results might be useful for sports governing bodies: we believe that FIFA could further increase the economic success of World Cups by using a more transparent and statistically validated method in the allocation of qualifying berths and the design of confederation-level qualification tournaments.

Acknowledgements

This paper could not have been written without my father (also called László Csató), who has primarily coded the simulations in Python.
We are grateful to Mark Gagolewski, Dries Goossens, Lars Hvattum, and Cristophe Ley for useful advice.
Five anonymous reviewers provided valuable comments and suggestions on earlier drafts.
We are indebted to the Wikipedia community for collecting and structuring valuable information on the sports tournaments discussed.
The research was supported by the MTA Premium Postdoctoral Research Program grant PPD2019-9/2019.

References

ABC, (2003) ABC (2003). Farina blasts FIFA World Cup backflip. Australian Broadcasting Corporation (ABC) News. 29 June. https://www.abc.net.au/news/2003-06-29/farina-blasts-fifa-world-cup-backflip/1878086.
Anbarcı et al., (2021) Anbarcı, N., Sun, C.-J., and Ünver, M. U. (2021). Designing practical and fair sequential team contests: The case of penalty shootouts. Games and Economic Behavior, 130:25–43.
Apesteguia and Palacios-Huerta, (2010) Apesteguia, J. and Palacios-Huerta, I. (2010). Psychological pressure in competitive environments: Evidence from a randomized natural experiment. American Economic Review, 100(5):2548–2564.
Arlegi and Dimitrov, (2020) Arlegi, R. and Dimitrov, D. (2020). Fair elimination-type competitions. European Journal of Operational Research, 287(2):528–535.
Baker and McHale, (2018) Baker, R. D. and McHale, I. G. (2018). Time-varying ratings for international football teams. European Journal of Operational Research, 267(2):659–666.
Brams and Ismail, (2018) Brams, S. J. and Ismail, M. S. (2018). Making the rules of sports fairer. SIAM Review, 60(1):181–202.
Cea et al., (2020) Cea, S., Durán, G., Guajardo, M., Sauré, D., Siebert, J., and Zamorano, G. (2020). An analytics approach to the FIFA ranking procedure and the World Cup final draw. Annals of Operations Research, 286(1-2):119–146.
Chater et al., (2021) Chater, M., Arrondel, L., Gayant, J.-P., and Laslier, J.-F. (2021). Fixing match-fixing: Optimal schedules to promote competitiveness. European Journal of Operational Research, 294(2):673–683.
CONCACAF, (2019) CONCACAF (2019). Concacaf Announces Format for the 2022 FIFA World Cup Confederation Qualifiers. 10 July. https://www.concacaf.com/en/world-cup-qualifying-men/article/concacaf-announces-format-for-the-2022-fifa-world-cup-confederation-qualifiers.
Corona et al., (2019) Corona, F., Forrest, D., Tena, J. D., and Wiper, M. (2019). Bayesian forecasting of UEFA Champions League under alternative seeding regimes. International Journal of Forecasting, 35(2):722–732.
(11) Csató, L. (2020a). The incentive (in)compatibility of group-based qualification systems. International Journal of General Systems, 49(4):374–399.
(12) Csató, L. (2020b). When neither team wants to win: A flaw of recent UEFA qualification rules. International Journal of Sports Science & Coaching, 15(4):526–532.
(13) Csató, L. (2021a). A comparison of penalty shootout designs in soccer. 4OR, 19(5):183–198.
(14) Csató, L. (2021b). Tournament Design: How Operations Research Can Improve Sports Rules. Palgrave Pivots in Sports Economics. Palgrave Macmillan, Cham, Switzerland.
Csató and Petróczy, (2022) Csató, L. and Petróczy, D. G. (2022). Fairness in penalty shootouts: Is it worth using dynamic sequences? Manuscript. arXiv: 2004.09225.
Dagaev and Rudyak, (2019) Dagaev, D. and Rudyak, V. (2019). Seeding the UEFA Champions League participants: Evaluation of the reform. Journal of Quantitative Analysis in Sports, 15(2):129–140.
Dagaev and Sonin, (2018) Dagaev, D. and Sonin, K. (2018). Winning by losing: Incentive incompatibility in multiple qualifiers. Journal of Sports Economics, 19(8):1122–1146.
Depetris-Chauvin et al., (2020) Depetris-Chauvin, E., Durante, R., and Campante, F. (2020). Building nations through shared experiences: Evidence from African football. American Economic Review, 110(5):1572–1602.
Deutsch, (2011) Deutsch, R. C. (2011). Looking back at South Africa: Analyzing and reviewing the 2010 FIFA World Cup. Chance, 24(2):15–23.
Durán et al., (2017) Durán, G., Guajardo, M., and Sauré, D. (2017). Scheduling the South American Qualifiers to the 2018 FIFA World Cup by integer programming. European Journal of Operational Research, 262(3):1109–1115.
Dyte and Clarke, (2000) Dyte, D. and Clarke, S. R. (2000). A ratings based Poisson model for World Cup soccer simulation. Journal of the Operational Research Society, 51(8):993–998.
FIFA, (2017) FIFA (2017). FIFA Council confirms contributions for FIFA World Cup participants. 27 October. http://web.archive.org/web/20200726200924/https://www.fifa.com/who-we-are/news/fifa-council-confirms-contributions-for-fifa-world-cup-participants-2917806.
(23) FIFA (2018a). 2026 FIFA World Cup^TM: FIFA Council designates bids for final voting by the FIFA Congress. 10 June. http://web.archive.org/web/20210306161039/https://www.fifa.com/who-we-are/news/2026-fifa-world-cuptm-fifa-council-designates-bids-for-final-voting-by-the-fifa-.
(24) FIFA (2018b). Revision of the FIFA / Coca-Cola World Ranking. https://img.fifa.com/image/upload/edbm045h0udbwkqew35a.pdf.
Flégl, (2014) Flégl, M. (2014). Performance analysis during the 2014 FIFA World Cup qualification. The Open Sports Sciences Journal, 7:183–197.
Football rankings, (2020) Football rankings (2020). Simulation of scheduled football matches. 28 December. http://www.football-rankings.info/2020/12/simulation-of-scheduled-football-matches.html.
Frawley and Van den Hoven, (2015) Frawley, S. and Van den Hoven, P. (2015). Football participation legacy and Australia’s qualification for the 2006 Football World Cup. Soccer & Society, 16(4):482–492.
Groh et al., (2012) Groh, C., Moldovanu, B., Sela, A., and Sunde, U. (2012). Optimal seedings in elimination tournaments. Economic Theory, 49(1):59–80.
Groll et al., (2021) Groll, A., Hvattum, L. M., Ley, C., Popp, F., Schauberger, G., Van Eetvelde, H., and Zeileis, A. (2021). Hybrid machine learning forecasts for the UEFA EURO 2020. Manuscript. arXiv: 2106.05799.
(30) Groll, A., Ley, C., Schauberger, G., and Van Eetvelde, H. (2019a). A hybrid random forest to predict soccer matches in international tournaments. Journal of Quantitative Analysis in Sports, 15(4):271–287.
(31) Groll, A., Ley, C., Schauberger, G., Van Eetvelde, H., and Zeileis, A. (2019b). Hybrid machine learning forecasts for the FIFA Women’s World Cup 2019. Manuscript. arXiv: 1906.01131.
Groll et al., (2015) Groll, A., Schauberger, G., and Tutz, G. (2015). Prediction of major international soccer tournaments based on team-specific regularized Poisson regression: An application to the FIFA World Cup 2014. Journal of Quantitative Analysis in Sports, 11(2):97–115.
Guyon, (2015) Guyon, J. (2015). Rethinking the FIFA World Cup^TM final draw. Journal of Quantitative Analysis in Sports, 11(3):169–182.
Guyon, (2018) Guyon, J. (2018). What a fairer 24 team UEFA Euro could look like. Journal of Sports Analytics, 4(4):297–317.
Guyon, (2020) Guyon, J. (2020). Risk of collusion: Will groups of 3 ruin the FIFA World Cup? Journal of Sports Analytics, 6(4):259–279.
Haugen and Krumer, (2021) Haugen, K. K. and Krumer, A. (2021). On importance of tournament design in sports management: Evidence from the UEFA Euro 2020 qualification. In Ratten, V., editor, Innovation and Entrepreneurship in Sport Management, pages 22–35. Edward Elgar Publishing, New York.
Hvattum and Arntzen, (2010) Hvattum, L. M. and Arntzen, H. (2010). Using ELO ratings for match result prediction in association football. International Journal of Forecasting, 26(3):460–470.
Jones, (1990) Jones, M. C. (1990). The World Cup draw’s flaws. The Mathematical Gazette, 74(470):335–338.
Krumer and Lechner, (2017) Krumer, A. and Lechner, M. (2017). First in first win: Evidence on schedule effects in round-robin tournaments in mega-events. European Economic Review, 100:412–427.
Krumer et al., (2017) Krumer, A., Megidish, R., and Sela, A. (2017). First-mover advantage in round-robin tournaments. Social Choice and Welfare, 48(3):633–658.
Krumer et al., (2020) Krumer, A., Megidish, R., and Sela, A. (2020). The optimal design of round-robin tournaments with three players. Journal of Scheduling, 23(3):379–396.
Laica et al., (2021) Laica, C., Lauber, A., and Sahm, M. (2021). Sequential round-robin tournaments with multiple prizes. Games and Economic Behavior, 129:421–448.
Laliena and López, (2019) Laliena, P. and López, F. J. (2019). Fair draws for group rounds in sport tournaments. International Transactions in Operational Research, 26(2):439–457.
Lambers and Spieksma, (2021) Lambers, R. and Spieksma, F. C. R. (2021). A mathematical analysis of fairness in shootouts. IMA Journal of Management Mathematics, 32(4):411–424.
Lasek et al., (2013) Lasek, J., Szlávik, Z., and Bhulai, S. (2013). The predictive power of ranking systems in association football. International Journal of Applied Pattern Recognition, 1(1):27–46.
Lasek et al., (2016) Lasek, J., Szlávik, Z., Gagolewski, M., and Bhulai, S. (2016). How to improve a team’s position in the FIFA ranking? A simulation study. Journal of Applied Statistics, 43(7):1349–1368.
Ley et al., (2019) Ley, C., Van de Wiele, T., and Van Eetvelde, H. (2019). Ranking soccer teams on the basis of their current strength: A comparison of maximum likelihood approaches. Statistical Modelling, 19(1):55–73.
O’Leary, (2017) O’Leary, D. E. (2017). Crowd performance in prediction of the World Cup 2014. European Journal of Operational Research, 260(2):715–724.
Palacios-Huerta, (2012) Palacios-Huerta, I. (2012). Tournaments, fairness and the Prouhet-Thue-Morse sequence. Economic Inquiry, 50(3):848–849.
Palacios-Huerta, (2014) Palacios-Huerta, I. (2014). Beautiful Game Theory: How Soccer Can Help Economics. Princeton University Press, Princeton.
Pollard and Armatas, (2017) Pollard, R. and Armatas, V. (2017). Factors affecting home advantage in football World Cup qualification. International Journal of Performance Analysis in Sport, 17(1-2):121–135.
Rathgeber and Rathgeber, (2007) Rathgeber, A. and Rathgeber, H. (2007). Why Germany was supposed to be drawn in the group of death and why it escaped. Chance, 20(2):22–24.
Sahm, (2019) Sahm, M. (2019). Are sequential round-robin tournaments discriminatory? Journal of Public Economic Theory, 21(1):44–61.
Scarf et al., (2009) Scarf, P., Yusof, M. M., and Bilbao, M. (2009). A numerical study of designs for sporting contests. European Journal of Operational Research, 198(1):190–198.
Scarf and Yusof, (2011) Scarf, P. A. and Yusof, M. M. (2011). A numerical study of tournament structure and seeding policy for the soccer World Cup Finals. Statistica Neerlandica, 65(1):43–57.
Stone and Rod, (2016) Stone, C. and Rod, M. (2016). Unfair play in World Cup qualification? An analysis of the 1998–2010 FIFA World Cup performances and the bias in the allocation of tournament berths. Soccer & Society, 17(1):40–57.
Van Eetvelde and Ley, (2019) Van Eetvelde, H. and Ley, C. (2019). Ranking methods in soccer. In Kenett, R. S., Longford, T. N., Piegorsch, W., and Ruggeri, F., editors, Wiley StatsRef: Statistics Reference Online, pages 1–9. Springer, Hoboken, New Jersey, USA.
Vandebroek et al., (2018) Vandebroek, T. P., McCann, B. T., and Vroom, G. (2018). Modeling the effects of psychological pressure on first-mover advantage in competitive interactions: The case of penalty shoot-outs. Journal of Sports Economics, 19(5):725–754.
Vaziri et al., (2018) Vaziri, B., Dabadghao, S., Yih, Y., and Morin, T. L. (2018). Properties of sports ranking methods. Journal of the Operational Research Society, 69(5):776–787.
Vong, (2017) Vong, A. I. K. (2017). Strategic manipulation in tournament games. Games and Economic Behavior, 102:562–567.

Appendix

Table A.1: CONCACAF nations participating in the 2018 FIFA World Cup qualification

Country	Elo rating of 21 March 2015
Costa Rica	1866
Mexico	1893
United States	1816
Honduras	1539
Panama	1706
Trinidad and Tobago	1525
Jamaica	1522
Haiti	1446
Canada	1507
Cuba	1396
Aruba	941
Dominican Republic	1235
El Salvador	1507
Suriname	1244
Guatemala	1539
Saint Vincent and the Grenadines	1162
Saint Lucia	1132
Grenada	1158
Antigua and Barbuda	1271
Guyana	1221
Puerto Rico	1059
Saint Kitts and Nevis	1204
Belize	1138
Montserrat	619
Dominica	951
Barbados	1155
Bermuda	1210
Nicaragua	1105
Turks and Caicos Islands	776
Curacao	1087
U.S. Virgin Islands	679
Bahamas	933
Cayman Islands	937
British Virgin Islands	612
Anguilla	575

•

The countries are ranked according to the FIFA World Ranking of August 2014. The first match in the first round was played on 22 March 2015.
•

The team(s) written in bold (italics) qualified for the 2018 FIFA World Cup (inter-confederation play-offs).

Table A.2: AFC nations participating in the 2018 FIFA World Cup qualification

Country	Elo rating of 11 March 2015
Iran	1724
Japan	1757
South Korea	1736
Australia	1727
United Arab Emirates	1666
Uzbekistan	1625
China	1606
Iraq	1509
Saudi Arabia	1474
Oman	1522
Qatar	1525
Jordan	1503
Bahrain	1487
Vietnam	1236
Syria	1478
Kuwait	1446
Afghanistan	1084
Philippines	1225
Palestine	1274
Maldives	1068
Thailand	1363
Tajikistan	1262
Lebanon	1403
Kyrgyzstan	1100
North Korea	1446
Myanmar	1062
Turkmenistan	1261
Indonesia	1207
Singapore	1173
Malaysia	1225
Hong Kong	1152
Bangladesh	952
Guam	806
Laos	837
India	1068
Sri Lanka	829
Yemen	1211
Cambodia	717
Chinese Taipei	795
Timor-Leste	639
Nepal	883
Macau	636
Pakistan	943
Mongolia	734
Brunei	606
Bhutan	521

•

The countries are ranked according to the FIFA World Ranking of January 2015 ( $35$ – $46$ ) and April 2015 ( $1$ – $34$ ). The first match in the first round was played on 12 March 2015.
•

The team(s) written in bold (italics) qualified for the 2018 FIFA World Cup (inter-confederation play-offs).

Table A.3: CONMEBOL nations participating in the 2018 FIFA World Cup qualification

Country	Elo rating of 7 October 2015
Argentina	2067
Colombia	1999
Brazil	2048
Chile	1977
Uruguay	1873
Ecuador	1822
Peru	1807
Paraguay	1696
Bolivia	1636
Venezuela	1668

•

The countries are ranked according to the FIFA World Ranking of October 2015. The first matchday was 8 October 2015.
•

The team(s) written in bold (italics) qualified for the 2018 FIFA World Cup (inter-confederation play-offs).

Table A.4: OFC nations participating in the 2018 FIFA World Cup qualification

Country	Elo rating of 30 August 2015
New Zealand	1516
New Caledonia	1357
Tahiti	1226
Solomon Islands	1175
Vanuatu	1174
Fiji	1281
Papua New Guinea	1053
Samoa	681
Tonga	684
American Samoa	381
Cook Islands	675

•

The countries are ranked according to the FIFA World Ranking of July 2015, separately for the positions $1$ – $7$ (byes in the first round) and $8$ – $11$ (playing in the first round). The first match in the first round was played on 31 August 2015.
•

The team written in italics qualified for the inter-confederation play-offs of the 2018 FIFA World Cup.

Quantifying the unfairness of the 2018 FIFA World Cup qualification