Fostering Cooperation in Structured Populations Through Local and Global Interference Strategies

We consider a population of agents on a square lattice of size $Z=L\times L$ with periodic boundary conditions— a widely adopted population structure in population dynamics and evolutionary games (for a survey, see szabo2007evolutionary). We focus our analysis on the efficiency of various interference strategies in spatial settings, adopting an agent-based model directly comparable with the setup of recent lab experiments on cooperation rand2014static.

Initially each agent is designated either as a cooperator (C) or defector (D) with equal probability. Agents’ interaction is modelled using the one-shot Prisoner’s Dilemma game, where mutual cooperation (mutual defection) yields the reward $R$ (penalty $P$) and unilateral cooperation gives the cooperator the sucker’s payoff $S$ and the defector the temptation $T$. As a popular interaction model of structured populations szabo2007evolutionary, we adopt the following scaled payoff matrix of the PD: $T=b$, $R=1$, $P=S=0$ ¹¹1The results obtained in this paper remain robust when $P$ is a small positive value, resulting in a strict PD nowak1992evolutionary. (with $1<b\leq 2$).

At each time step or generation, each agent plays the PD with its (four) immediate neighbours. The score for each agent is the sum of the payoffs in these encounters. At the start of the next generation, each agent’s strategy is changed to that of its highest scored neighbour nowak1992evolutionary; szabo2007evolutionary. Our analysis will be primarily based on this deterministic, standard evolutionary process in order to focus on understanding the cost-efficiency of different interference strategies. However, we confirm that all our conclusions remain valid for a stochastic update rule (see Section 4.4 for more details). Namely, instead of coping the highest scored neighbour, at the end of each generation an agent $A$ with score $f_{A}$ chooses to copy the strategy of a randomly selected neighbour agent $B$ with score $f_{B}$ with a probability given by the Fermi rule traulsen2006stochastic: $(1+e^{(f_{A}-f_{B})/K})^{-1}$, where $K$ denotes the amplitude of noise in the imitation process szabo2007evolutionary. Varying K allows capturing a wide range of update rules and levels of stochasticity, including those used by humans, as measured in lab experiments zisisSciRep2015; randUltimatum.

We simulate this evolutionary process until a stationary state or a cyclic pattern is reached. Similarly to nowak1992evolutionary, all the simulations in this work (described in next section) converge quickly to such a state. For the sake of a clear and fair comparison, all simulations are run for 200 generations. Moreover, for each simulation, the results are averaged from additional 50 generations after that. Furthermore, to improve accuracy, for each set of parameter values, the final results are obtained from averaging 50 independent realisations.

Note that we do not consider mutations or explorations in this work. Thus, whenever the population reaches a homogeneous state (i.e. when the population consists of 100% of agents adopting the same strategy), it will remain in that state regardless of interference. Hence, whenever detecting such a state, no further interference will be made. It is noteworthy that the results remain robust assuming a sufficiently small mutation rate, allowing the population to reach a stable state before any new mutants arise.

As already stated, we aim to study how one can efficiently interfere in a structured population to achieve high levels of cooperation while minimising the cost of interference. An investment in a cooperator consists of a cost $\theta>0$ (to the external decision-maker/investor). In particular, we investigate whether global interference strategies (where investments are triggered based on network level information) or their local counterparts (where investments are based on local neighbourhood information) lead to successful behaviour with better cost efficiency. To do so, we consider two main classes of interference strategies based on the global composition of the population and the neighbourhood information.
1. Population composition based (POP): In this class of strategies the decision to interfere (i.e. to invest on all cooperators in the population) is based on the current composition of the population (we denote $x_{C}$ the number of cooperators currently in the population). Namely, they invest when the number of cooperators in the population is below a certain threshold, $p_{C}$ (i.e. $x_{C}\leq p_{C}$), for $1\leq p_{C}\leq Z$. They do not invest otherwise (i.e. when $x_{C}>p_{C}$). The value $p_{C}$ describes how widespread defection strategy should be to trigger the support of cooperators’ survival against defectors.

In addition to these primary strategy classes, we consider two additional advanced local strategies, assuming that the external decision-maker can assess individual scores of players in a neighbourhood (e.g. an institution might have access to individuals’ income records).
(NEB-i) Maintaining-C: for every cooperator, if the best scoring neighbour is a D which has a payoff larger than that of C, then invest an amount equal to the difference of the payoff of best scoring neighbour to that of C, plus a small amount $\epsilon>0$.
(NEB-ii) Maintaining-C plus influencing D-neighbours. For every C we proceed as in (NEB-i). We then perform an additional second step in which we lure D neighbours of C to copy its strategy. To achieve this, if the highest scoring neighbour of D is a defector, then invest an additional amount to ensure that C has a higher payoff than that of the most fit D, by a small amount $\epsilon>0$.

Note that these interference strategies are increasingly more subtle, requiring more information and so more detailed observations of the population. In particular, POP requires knowledge of overall cooperativeness of the population, while NEB requires information about local cooperativeness in each neighbourhood. NEB-i and NEB-ii need to access the fitness levels of all players in each neighbourhood.

We compare the population-based and neighbourhood based interference strategies, i.e. POP vs NEB, namely their abilities to promote cooperation while maintaining a minimal total cost. In Figure 1, we investigate the evolution over time of the frequency of cooperation and the corresponding cost required as per-generation for POP and NEB strategies, and for different values of $p_{C}$ and $n_{C}$, respectively. We observe that for POP, the population converges to cyclic patterns for a small $p_{C}$, while when $p_{C}$ is sufficiently large (close to 100% in general), the population quickly converges to 100% of cooperation, avoiding cyclic behaviours. Cyclic patterns occur since interference can lead to an increase in the frequency of Cs; when this rate becomes larger than $p_{C}$, it causes the interference by POP to stop, which in turn leads to the decrease of the frequency of cooperative actions and then causes interference to restart.

On the other hand, for NEB, when $n_{C}\leq 2$, the population converges to a stable level of cooperation nurtured by constant interference. When $n_{C}\geq 3$, the population quickly converges to 100% of cooperation, thus avoiding the cost of maintaining a stable state. Most interestingly, comparing $n_{C}=4$ and $n_{C}=3$, we observe that the per-generation cost increases for the former until reaching 100% of cooperation, while it decreases for the latter. The reason is that there are increasingly more cooperators requiring costly investment in the former case while such a number decreases in the latter. As such, NEB with $n_{C}=3$ has a significantly lower total cost than NEB with $n_{C}=4$ while guaranteeing similar high levels of cooperation (see Figure 4 for other parameter values of $\theta$).

Next, for each interference approach (POP and NEB) and threshold ($p_{C}$ and $n_{C}$), we relate the stationary cooperation level with the cost spent per-individual ($\theta$) (see Figure 2). In general, the C frequency obtained by POP mostly depends on $p_{C}$ – it requires a high $p_{C}$ ($>85\%$ of the whole population) to reach a high level of cooperation, turning POP a costly strategy. For NEB, similar high levels of cooperation can be achieved when $n_{C}\geq 3$ (comparing panels A and C). Considering the total cost minimisation (panels B and D), NEB has a significantly larger area where a low cost is required to achieve high levels of cooperation when compared to POP. Thus, adopting a local interference strategy is largely more efficient than a global population-wide paradigm.

From Figure 2, we also observe that both POP and NEB strategies require a certain threshold of $\theta$ to be successful. Next, we derive a closed-form expression for such critical thresholds, for arbitrary $b$.

From Figure 2B we observe that, for POP to obtain an efficient cost while ensuring high levels of cooperation (the black area near top right corner), $\theta$ must be sufficiently high. Our analysis of Figure 3A shows that whenever $\theta\leq 4b-3$, different cyclic patterns or stable mixed configurations are eventually reached. Particularly, when $\theta$ is slightly less than $4b-3$, the population always ends up in a stable pattern, characterised by a number of standalone D individuals surrounded by C neighbours. When $\theta>4b-3$ (i.e. when $b=1.8$, $\theta>4.2$), cyclic patterns can be escaped, leading to a small total cost and high levels of cooperation.

Similarly, from Figure 2D, for NEB to obtain an efficient cost while ensuring high levels of cooperation (the black area near top right corner), $\theta$ must be sufficiently high. We have derived the threshold of $\theta$ for $n_{C}=3$: $\theta\geq 4b-2$ (i.e. when $b=1.8$, $\theta\geq 5.2$), see Figure 3 (panels B and C). Moreover, since NEB with $n_{C}=4$ is equivalent to POP with $p_{C}=Z$ — both mean always invest in all cooperators — the same condition is obtained for these two limiting cases (i.e. $\theta>4b-3$).

We can see that the threshold of $\theta$ for NEB with $n_{C}=4$ is lower than the one obtained for NEB with $n_{C}=3$ ($4b-3$ compared to $4b-2$), which is also confirmed by the numerical results in Figure 2D. When $\theta$ lies between $4b-3$ and $4b-2$, NEB with $n_{C}=4$ is more cost-efficient.

We confirm by performing additional simulations that the two conditions hold for other values of $b$ and are also robust for different population sizes $Z$.

We compare the cost efficiency of NEB with $n_{C}=3$ and then with $n_{C}=4$ where the conditions for their efficiency (derived above) are satisfied, namely, $\theta\geq 4b-2$ and $\theta>4b-3$, respectively. Abusing notations, let us assume that $\theta=(4b-2)+\epsilon$ and $\theta=(4b-3)+\epsilon$ for the two cases, respectively. This allows us to conveniently compare these strategies with the more advanced ones, NEB-i and NEB-ii. The parameter $\epsilon$ represents the extra amount to the threshold value of $\theta$ which guarantees (close to) 100% of cooperation.

Figure 4 shows the total costs for the four interference strategies for varying $\epsilon$. They all guarantee high levels of cooperation (100%) for $\epsilon>0$. In general, NEB-3 is always significantly more cost-efficient than that with $n_{C}=4$. Moreover, assuming that it is possible to assess individual scores of players in a neighbourhood, more cost-efficient strategies are obtained. Namely, both NEB-ii and NEB-i are slightly better than NEB-3.

We confirm that our results are robust with respect to the stochastic update rule (see again Section 3.1). In particular, similarly to the deterministic case, we observe that the local interference approach (NEB) leads to more cost-efficient strategies than the global interference one (POP). Namely, NEB with $n_{C}=3$ is highly cost-efficient (while guaranteeing high levels of cooperation) – see Figure 5 for a typical evolution of strategies and per-generation costs (and the total costs) for NEB with $n_{C}=3$ and POP with $p_{C}=Z$. Interestingly, due to the stochastic effect (where there is a chance that a higher scored individual copies the lower scored neighbour’s strategy), it is easier to escape cyclic configurations and reach full cooperation in the population. As a result, a lower threshold of $\theta$ is required to reach good performance of the interference strategies (e.g. in Figure 5, $\theta=3$ was sufficient to reach full cooperation, compared to $\theta\geq 5.2$ in the deterministic case). However, due to this stochastic effect, it takes longer to converge to a homogeneous state (compared Figures 5 and 1 where $n_{C}=3$ and $n_{C}=4$, noting that NEB with $n_{C}=4$ is equivalent to POP with $p_{C}=Z$).