Evolutionary stability of cooperation in indirect reciprocity under noisy and private assessment

Cooperation benefits others but is costly to the cooperator itself. Nevertheless, cooperation is widespread from microscopic to macroscopic scales, such as among microorganisms, animals, humans, and nations. One way to sustain cooperation is that agents conditionally cooperate with others who cooperate with them, which is realized by, for example, repeated interactions [1, 2, 3] and partner choice [4, 5, 6, 7]. Such conditional cooperation based on personal experiences is applicable only to a small population where members can interact directly and repeatedly with most of the others.

However, cooperative behavior is observed even in a large-scale society (e.g., human societies). Since individuals inevitably encounter strangers there, they need reputations of those strangers in order not to cooperate unconditionally. Only individuals with good reputations can receive cooperation. The system that individuals indirectly reward others via their reputations as described above is called indirect reciprocity [8, 9, 10]. In reality, humans are particularly interested in reputations and gossip about themselves and others [11, 12, 13]. Furthermore, many experiments have pointed out that gossips concern cooperative behaviors [14, 15, 16].

Errors that inevitably occur in actions and in assessment hinder cooperation by indirect reciprocity. Indeed, the simplest social norm called image scoring [9, 10] fails to maintain full cooperation under errors [17, 18] (a similar failure is also seen in direct reciprocity [19, 20, 18]). This is because one erroneous defection triggers further defection. Nevertheless, previous studies have theoretically shown that cooperation can be maintained by the so-called “leading eight” social norms [21, 22] even in the presence of such errors when all individuals share the reputation of the same individual (i.e., public assessment). Public reputation cases have been thoroughly studied for about two decades [23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37]. When individuals cannot share their evaluations of the same target (i.e., private assessment), however, errors cast a shadow over cooperation more crucially. In this case, a single disagreement in opinions between two individuals can lead to further disagreements [38, 39, 40, 41, 42]. Whether cooperation is maintained under the noisy and private assessment is still largely unsolved in theory and is one of the major open problems in studies of indirect reciprocity [43, 44, 36].

Previous studies have shown that maintaining cooperation with indirect reciprocity is very difficult under noisy and private assessment. For example, Hilbe et al. [42] showed by an evolutionary simulation that the above leading eight strategies cannot succeed in cooperation under private assessment. Some studies [45, 46, 47] have demonstrated the emergence of cooperation under noisy and private assessment, but under the restrictive assumption that only local mutations in the strategy space are allowed, thus excluding the possibility that a fully cooperative strategy is directly invaded by free-riders. Other studies have shown that a mechanism to synchronize opinions between individuals has a positive influence on cooperation in indirect reciprocity, such as empathy, generosity, spatial structure, and so on [48, 49, 50, 51, 52, 53, 54, 55, 56, 57].

Most of these studies of private assessment have been performed by computer simulations [45, 42]. This is because two-dimensional information of who assigns a reputation to whom (its matrix representation is called “image matrix” [38, 58, 39, 59]) becomes too complex to analyze. For example, its possible transition is illustrated in Fig. 1-A, where a single assessment error can be amplified with time, leading to a mosaic structure in the image matrix. An evolutionary analysis between wild-type and mutant makes the image matrix further complex because the image matrix now includes four compartments based on different rules of reputation assignment adopted by wild-type and mutant individuals (Fig. 1-B). In spite of these difficulties, here we report that we have successfully developed an analytical machinery to study the image matrix by applying a technique previously developed by the authors [60]. This enables us to make a general prediction of when cooperation is sustained under noisy and private assessment over the full parameter region.

In the following, we will first introduce the setting of indirect reciprocity under noisy and private assessment and explain a method to analytically calculate the expected payoff of each individual through analyzing a complex image matrix. Then, we will discuss which strategy can be an evolutionarily stable strategy (ESS) [61, 62] under which condition, and provide intuitive reasons for the result. To our knowledge, this is the first systematic study that has analytically investigated evolutionary stability of strategies in indirect reciprocity under noisy and private assessment.

We consider a model of indirect reciprocity in a well-mixed population of size $N$. We assume that, in every step, a binary reputation is assigned independently from everyone to everyone, either good or bad, which is summarized by image matrix $\{\beta_{ji}\}$, where $\beta_{ji}=1$ (resp. $\beta_{ji}=0$) if individual $i$ assigns a good (resp. bad) reputation to individual $j$. The model proceeds as follows. First, a donor and a recipient are randomly chosen from this population. Next, the donor takes its action, cooperation or defection, to the recipient. When the donor cooperates, the donor incurs a cost $c(>0)$ but gives a benefit $b(>c)$ to the recipient instead. On the other hand, when the donor defects, no change occurs in the payoff of the donor or the recipient. Here, a rule that specifies how the donor chooses its action is called “action rule”. Throughout this paper, we assume that all the individuals adopt the “discriminator” action rule [9, 63], with which they choose cooperation (resp. defection) to a good (resp. bad) recipient in their own eyes; that is, donor $i$ chooses cooperation toward recipient $j$ if $\beta_{ji}=1$, and chooses defection if $\beta_{ji}=0$. We assume that the donor unintentionally takes the opposite action to the intended one with probability $0\leq e_{1}<1/2$ (action error). All the individuals in the population observe this social interaction between the donor and the recipient and independently update the reputation of the donor in their eyes.

A rule that specifies how each observer updates the reputations of the others is called its “social norm”. In models of public reputation, it has often been assumed that all the individuals in the population adopt the same social norm [21, 64, 29] (but see [38]), otherwise, they cannot share the reputation of the same individual. Because we consider a model of private reputation here, however, we instead assume that individuals can adopt different social norms. This study deals with a situation where each observer (say, $k$) refers to (i) whether the donor (say, $i$) cooperates (C) or defects (D) (first-order information) and (ii) whether the recipient (say, $j$) is good (G) or bad (B) in the eyes of the observer (second-order information, represented by $\beta_{jk}$) when this observer updates the reputation of the donor in the eyes of the observer, denoted by $\beta_{ik}$. Such social norms are called “second-order” social norms [10, 18, 33, 36]. An observer who adopts a second-order social norm can face four different cases, denoted by GC(“toward a Good recipient the donor Cooperates”), BC(“toward a Bad recipient the donor Cooperates”), GD(“toward a Good recipient the donor Defects”), and BD(“toward a Bad recipient the donor Defects”), respectively, and in each case, the observer assigns either a good (G) or bad (B) reputation to the donor. Thus, a social norm is represented by a four-letter string. For example, $\mathrm{GBBG}$ is the social norm that assigns to the donor a good reputation in GC- and BD-cases, and a bad reputation in BC- and GD-cases. There are $2^{4}=16$ such social norms in total, and we lexicographically order them with the rule that G comes first and B comes second, and number them from $S_{01}$ to $S_{16}$. Table 1 shows a full list of 16 social norms studied here. When updating the reputation, each observer independently commits an assessment error with probability $0<e_{2}<1/2$, in which case he/she accidentally assigns the opposite reputation to the intended one to the donor.

Several norms are especially important in previous studies, so we explain them below. We call $S_{01}$ ALLG and call $S_{16}$ ALLB because these norms unconditionally assign good or bad reputations. Next, $S_{03}$, $S_{04}$, $S_{07}$, and $S_{08}$ belong to G$\ast$B$\ast$ family. These norms share the same feature that they regard cooperation toward a good recipient as good, and defection toward a good recipient as bad. They only differ when the recipient is bad in the observer’s eyes. First, $S_{04}$ is called Scoring (SC), which regards cooperation toward a bad recipient as good and defection toward a bad recipient as bad, and therefore reputation assignment is independent of whether the recipient is good or bad in the observer’s eyes (thus, categorized as a first-order norm). Next, $S_{07}$ is called Stern Judging (SJ), which regards cooperation toward a bad recipient as bad and defection toward a bad recipient as good, as opposed to SC. Third, $S_{03}$ is called Simple Standing (SS) and it regards any action toward a bad recipient as good, and therefore it is the most generous norm in this family. Finally, $S_{08}$ is called Shunning (SH) and it regards any action toward a bad recipient as bad, and therefore it is the most intolerant one. Notably, SJ and SS are the two second-order norms that are included in the “leading eight” norms [21], which are norms that can successfully maintain cooperation under the noisy and public reputation that are found in the search within third-order norms. In particular, SJ has long been considered promising because it is evolutionarily successful [23] and because it sustains a very high level of cooperation despite its simplicity [33, 36]. SJ always suggests only one correct action to keep you good; it recommends cooperation toward good individuals and defection toward bad ones, and failure to follow this rule leads to a bad reputation. Under the noisy public reputation, SH cannot achieve full cooperation against itself but can prevent the invasion of ALLB (see SI for detailed calculation).

Under these settings, the strategy of an individual is its social norm. For this reason, we use “strategy” and “(social) norm” interchangeably in the following. We ask which strategy is evolutionarily stable. To this end, we study invasibility of a mutant strategy against a wild-type one. A strategy is ESS if it is not invaded by any other 15 mutant strategies. To derive their payoffs, we need to analyze the image matrix, which we shall perform below.

Let us consider a situation where individuals with mutant norm ${\rm M}$ invade the population of wild-type norm ${\rm W}(\neq{\rm M})$. Here, the proportion of mutants is given by $\delta$. By extending the Fujimoto & Ohtsuki’s method [60] we can describe the image matrix by two probability distributions. Specifically, take a focal individual whose norm is $A\in\{{\rm W},{\rm M}\}$, and let $p_{AA^{\prime}}$ (hereafter called “goodness”) be the proportion of individuals among norm $A^{\prime}$ users who assign a good reputation to the focal individual, for $A^{\prime}\in\{{\rm W},{\rm M}\}$. Thus, a wild-type individual is characterized by a pair of goodnesses, $(p_{\mathrm{WW}},p_{\mathrm{WM}})$, and we represent its distribution over all wild-type individuals by $\Phi_{\rm W}(p_{\mathrm{WW}},p_{\mathrm{WM}})$. In the same way, a mutant is characterized by a pair of goodnesses, $(p_{\mathrm{MW}},p_{\mathrm{MM}})$, and $\Phi_{\rm M}(p_{\mathrm{MW}},p_{\mathrm{MM}})$ represents its distribution over all mutants. In SI, we derive the dynamics of $\Phi_{\rm W}$ and $\Phi_{\rm M}$ by formulating a stochastic transition of the donor’s goodnesses under the assumption of $N\gg 1$ (the population is large), $\delta\ll 1$ (mutants are rare), and $N\delta\gg 1$ (yet the number of mutants is sufficiently large). Then, we derive the equilibrium distributions, $\Phi_{\rm W}^{*}$ and $\Phi_{\rm M}^{*}$. These equilibrium distributions give expected payoffs of wild-types and mutants, which enable us to study the invasibility condition of mutants to wild-types (see SI again).

We find that each of the two equilibrium distributions is well approximated by a weighted sum of two-dimensional Gaussian functions with zero covariance, where each Gaussian can be systematically labeled by a nonzero integer, $j\in\mathbb{Z}\backslash\{0\}$ (see an example in Fig. 2-B and the rule of labeling in Fig. 2-C). Hence the number of Gaussians that appear in the sum is infinitely but countably many. In some cases, however, these labels degenerate (i.e., two or more Gaussians are identical but they are given different labels) and the number of Gaussians can be finite. Weights to Gaussians decay exponentially as $j$ becomes large positive or large negative, so a truncation at some finite number of terms approximates well the infinite sum for numerical calculations.

Based on the analysis of the image matrix above, we have studied pairwise invasibility for all the pairs of wild-type $\rm W$ and mutant $\rm M$. In the following, we set the action error rate as $e_{1}=0$, because this error, especially when it is small positive, does not have a qualitative impact on our results as far as we studied. Thus, the cost-benefit ratio $b/c$ and the assessment error rate $e_{2}$ are our environmental parameters.

We first find that the four strategies, $S_{06}$, $S_{07}$(SJ), $S_{10}$, and $S_{11}$, are completely indistinguishable, both as wild-types and as mutants. This is because these norms always give the goodness of $1/2$ to anyone in the population at equilibrium due to an accumulation of assessment errors and hence they appear to choose cooperation and defection in a random manner. In particular, they are neutral to each other. For these reasons, we will discuss only $S_{07}$(SJ) as a representative of them and exclude the other three in the following analysis.

Our exhaustive analysis demonstrates that only three norms, $S_{03}$(SS), $S_{08}$(SH), and $S_{16}$(ALLB) can be ESS, and all the others cannot. As shown in Fig. 3-A, ALLB is ESS independent of $b/c$ and $e_{2}$, because it is the norm that assigns a bad reputation to everyone, saves the own cost, and provides no benefit to others. On the other hand, SS and SH achieve ESS for some $b/c$ and $e_{2}$; there are upper and lower bounds of $b/c$ for them to be ESS, which depend on $e_{2}$. Below we will look at its details.

The ESS condition of $S_{03}$(SS) is shown in Fig. 3-B. When $b/c$ exceeds the upper bound, the norm is invaded by $S_{01}$(ALLG) (compare the right and center panels of Fig. 3-A). On the other hand, when $b/c$ falls below the lower bound, the norm is invaded by $S_{04}$(SC) (compare the left and center panels of Fig. 3-A). Fig. 3-C shows that these theoretical bounds are also supported by individual-based simulations. Notably, the smaller $e_{2}$ is, the wider the ESS region of $S_{03}$(SS) becomes.

The ESS region of $S_{08}$(SH) is quite narrow in comparison to that of $S_{03}$(SS), as seen in Fig. 3-B. In addition, when $b/c$ exceeds the upper bound or falls below the lower bound, the norm is invaded by $S_{04}$(SC) and $S_{16}$(ALLB), respectively. The range of $b/c$-ratios that make SH evolutionarily stable is the widest at an intermediate $e_{2}$ (about $0.1$).

In contrast to these results, we find that $S_{07}$(SJ), which is known to be a successful norm when reputation is public, is invaded by norms such as $S_{16}$(ALLB) and $S_{08}$(SH) independent of the value of $b/c$ (and also independent of $e_{2}$), and therefore that it is never an ESS. This is summarized in Fig. 3-A.

To summarize, SS, SH, and SJ are all the ESS norms under the public assessment, but whether they remain ESS in the private assessment critically differs. This difference is clearly understood by focusing on how the reputation structure they give differs between the public and private reputation cases under a sufficiently small but positive assessment error rate, $e_{2}\ll 1$. Let us consider below, for example, whether each norm can prevent the invasion of ALLB, a potential invader norm.

Success of Simple Standing: The reputation structure that $S_{03}$(SS) gives differs little between the public and private reputation cases (see Fig. 3-D). Under the public reputation (see SI for the calculation), SS assigns good reputations to SS themselves (represented by the bar at goodness $=1$ in the top-left panel in Fig. 3-D), while bad reputations to ALLB (represented by the bar at goodness $=0$ in the top-right panel in Fig. 3-D). Thus, SS distinguishes between SS itself and the invader ALLB and prevents the invasion of ALLB. Even under the private reputation, SS still assigns good reputations to SS themselves (see the bottom-left panel in Fig. 3; high goodness of $1-e_{2}$ are given to the fraction $1-2e_{2}$ of SS individuals, for example), and assigns bad reputations to ALLB (see the bottom-right panel in Fig. 3; low goodness of $2e_{2}$ are given to the fraction $1-3e_{2}$ of ALLB individuals, for example). Thus, the distinction between SS and ALLB is maintained. For that reason, SS succeeds in achieving ESS even under the private assessment. The cooperation rate at this ESS is as high as $1-2e_{2}$ for small $e_{2}$, so it entails nearly perfect cooperation.

Failure of Stern Judging: Contrary to SS, the reputation structure that $S_{07}$(SJ) gives extremely differs between the public and private reputation cases (see Fig. 3-E). Under the public reputation (see SI for the calculation), wild-type SJ gives high goodness to other SJ (top-left in Fig. 3-E) and wild-type SJ gives low goodness to ALLB (top-right in Fig. 3-E). Thus, SJ prevents the invasion of ALLB. Under the private assessment, however, SJ gives goodness of 1/2 to other SJ individuals (bottom-left in Fig. 3-E) [39] while SJ gives goodness of 1/2 to ALLB individuals as well (bottom-right in Fig. 3-E). Thus, the distinction between SJ and ALLB is lost. This is why SJ fails to be ESS under the private assessment.

Shunning can be ESS, but the level of cooperation is low: We can understand why $S_{08}$(SH) achieves ESS only in a narrow region under private reputation (see SI for the detailed calculation and see Fig. S3 for the illustration for easy interpretation). Under the public reputation, SH gives good reputations to the half of other SH and bad reputations to the other half (top-left in Fig. S3) while SH gives bad reputations to almost all ALLB (top-right in Fig. S3). Thus, SH prevents the invasion of ALLB. Under private reputation, on the other hand, SH gives low goodness to both SH and ALLB (bottom-left and bottom-right in Fig. S3). Here, however, SH has a slightly better chance to receive good reputations than ALLB, in the order of $e_{2}^{2}$. This explains why SH prevents the invasion from ALLB only in a narrow region and also explains why its ESS condition becomes more strict for a smaller assessment error rate, $e_{2}$. The cooperation rate at a realized ESS is as low as $e_{2}$ for small $e_{2}$, so we conclude that $S_{08}$(SH) does not contribute to cooperation.

This study considered indirect reciprocity under noisy and private assessment. We focused on goodness of individual (i.e., what proportion of individuals gives the individual good reputations) between different norms and developed an analytical method to calculate the distribution of goodness at equilibrium. Using this methodology we studied whether a mutant norm succeeds in the invasion into a wild-type norm. Although both $S_{03}$(SS) and $S_{07}$(SJ) can be ESS under public reputation, we found that their evolutionary stability is totally different under private assessment. In particular, we found that $S_{03}$(SS) remains to be ESS under private assessment if the assessment error rate is small, while $S_{07}$(SJ) cannot be ESS no matter how small the error rate is.

The reason for this difference between $S_{03}$(SS) and $S_{07}$(SJ) comes from the difference in the complexity of these two norms. In the world of private assessment, errors in assessment accumulate independently among observers, which is a potential source of collapse of cooperation in the population. However, since $S_{03}$(SS) regards a cooperating donor as good no matter whether the recipient is good or bad, a discrepancy in the opinion toward the recipient between two different observers does not produce further discrepancy; those two observers can agree that such a cooperating donor is good. In contrast, $S_{07}$(SJ) is more complex than $S_{03}$(SS) and recipient’s reputation is always decisive information (see Table 1), so this complexity becomes an obstacle for correcting discrepancy between observers.

Hilbe et al. [42] studied by computer simulations whether the leading eight norms can sustain cooperation under the noisy and private assessment. They concluded that $S_{03}$(SS) (referred to as “L3” in their paper) and $S_{07}$(SJ) (“L6”) fail to achieve cooperation, which is contrary to our result. This difference is because we studied evolutionary stability in a deterministic model, while they studied fixation probability in a stochastic model. Because those two criteria are different, drawing a general conclusion is difficult, but the significance of our study lies in that we have shown that cooperation can be evolutionarily sustained even under private assessment.

A future direction of this study would be to examine ESS conditions of social norms when some of the assumptions are changed. For example, we have assumed second-order norms, in which individuals refer to a donor’s action (first-order information) and a recipient’s reputation (second-order one) when they update the donor’s reputation. However, humans may use more complex norms than the second-order ones. Studying the effect of higher-order information  [65, 32, 33, 36], such as the previous reputation of the donor (third-order information), would further deepen our understanding. We have also assumed that all individuals simultaneously update their opinions toward the same donor. However, in a real society, the number of people who can observe a single person’s behavior is limited. Thus, the effect of asynchronous updates of reputations is worth studying. Last but not least, we have implicitly assumed that game interactions last sufficiently long so that we can use equilibrium distributions of goodness for calculating payoffs (i.e. discount factor is $1$). However, the effect of initial reputation cannot necessarily be ignored in some cases.

In conclusion, we have demonstrated that cooperation can be evolutionarily stable even under the noisy and private assessment. Specifically, we have shown that Stern Judging, which is one of the most leading norms under public reputation, cannot distinguish between cooperators and defectors under private assessment and thus fails to achieve ESS. On the other hand, we have revealed that Simple Standing can be stable in a wide range of parameters. Based on these results, we predict that Simple Standing should play a key role in sustaining cooperation by indirect reciprocity under noisy private assessment. These findings provide a rigid theoretical basis for understanding human cooperation and pave the way for future studies in biology, psychology, sociology, and economics.

Y.F. acknowledges the support by JSPS KAKENHI Grant Number JP21J01393. H.O. acknowledges the support by JSPS KAKENHI Grant Number JP19H04431.

This section proposes an analytical method to obtain the reputation structure under indirect reciprocity. We assume a situation where rare mutants with norm ${\rm M}$ of ratio $\delta$ invade other wild-types with norm ${\rm W}$ of ratio $1-\delta$. We denote the population ratio of norm $A\in\{{\rm W},{\rm M}\}$ as $\rho_{A}$; $\rho_{A}=1-\delta$ when $A={\rm W}$, while $\rho_{A}=\delta$ when $A={\rm M}$. To characterize the reputation structure, we define $p_{iA}$ as a proportion of individuals of norm $A$ who assign good reputations to individual $i$. We call $p_{iA}$ a goodness of individual $i$ from norm $A\in\{{\rm W},{\rm M}\}$.

In the following, let us consider a stochastic transition of $p_{iA}$ in each round. In a single round, a recipient and a donor are chosen and labeled as $i_{\rm R}$ and $i_{\rm D}$, respectively. In this round, $p_{i_{\rm D}A_{\rm O}}$, i.e., the goodness of donor from norm $A_{\rm O}$, changes into the next goodness $p_{i_{\rm D}A_{\rm O}}^{\prime}$ for all $A_{\rm O}\in\{{\rm W},{\rm M}\}$. Below, we formulate the stochastic change separately for cases that the donor chooses to cooperate or defect.

C-map case: First, we consider a case that the donor cooperates with the recipient, occurring with a probability of

$\displaystyle h(p_{i_{\rm R}A_{\rm D}}):=p_{i_{\rm R}A_{\rm D}}(1-e_{1})+(1-p_{i_{\rm R}A_{\rm D}})e_{1}.$

(1)

In this case, $N\rho_{A_{\rm O}}p_{i_{\rm D}A_{\rm O}}^{\prime}$, i.e., the number of observers with norm $A_{\rm O}$ who give good reputations to the donor in the next round, follows a probability distribution of

		$\displaystyle N\rho_{A_{\rm O}}p_{i_{\rm D}A_{\rm O}}^{\prime}\sim N_{1}+N_{2},$		(2)
		$\displaystyle N_{1}\sim{\cal B}(N\rho_{A_{\rm O}}p_{i_{\rm R}A_{\rm O}},a^{\rm GC}_{A_{\rm O}}),$		(3)
		$\displaystyle N_{2}\sim{\cal B}(N\rho_{A_{\rm O}}(1-p_{i_{\rm R}A_{\rm O}}),a^{\rm BC}_{A_{\rm O}}).$		(4)

Here, ${\cal B}(n,a)$ denotes a binomial distribution with success probability $a$ and trial number $n$. In addition, $a^{XY}_{A_{\rm O}}$ denotes the probability that an observer with norm $A_{\rm O}$ who evaluates the recipient as $X\in\{{\rm G},{\rm B}\}$ newly gives a good reputation to the donor whose action is $Y\in\{{\rm C},{\rm D}\}$. $a^{\rm GC}_{A_{\rm O}}$, $a^{\rm BC}_{A_{\rm O}}$, $a^{\rm GD}_{A_{\rm O}}$, and $a^{\rm BD}_{A_{\rm O}}$ are obtained by converting corresponding $\rm G$ and $\rm B$ pivots into $1-e_{2}$ and $e_{2}$ in Table 1 of the main manuscript. Instead of (3), we use a shorthand notation;

$\displaystyle N\rho_{A_{\rm O}}p_{i_{\rm D}A_{\rm O}}^{\prime}\sim{\cal B}(N\rho_{A_{\rm O}}p_{i_{\rm R}A_{\rm O}},a^{\rm GC}_{A_{\rm O}})+{\cal B}(N\rho_{A_{\rm O}}(1-p_{i_{\rm R}A_{\rm O}}),a^{\rm BC}_{A_{\rm O}}).$

(5)

Because $N\rho_{A_{\rm O}}$ is sufficiently large, the mean and variance of $p_{i_{\rm D}A_{\rm O}}^{\prime}$ are given by

	$\displaystyle\textrm{E}[p_{i_{\rm D}A_{\rm O}}^{\prime}]$	$\displaystyle=p_{i_{\rm R}A_{\rm O}}(\underbrace{a^{\rm GC}_{A_{\rm O}}-a^{\rm BC}_{A_{\rm O}}}_{=:\Delta f_{A_{\rm O}}^{\rm C}})+a^{\rm BC}_{A_{\rm O}}\quad(=:f_{A_{O}}^{\rm C}(p_{i_{\rm R}A_{\rm O}})),$		(6)
	$\displaystyle\textrm{Var}[p_{i_{\rm D}A_{\rm O}}^{\prime}]$	$\displaystyle=\frac{p_{i_{\rm R}A_{\rm O}}a^{\rm GC}_{A_{\rm O}}(1-a^{\rm GC}_{A_{\rm O}})+(1-p_{i_{\rm R}A_{\rm O}})a^{\rm BC}_{A_{\rm O}}(1-a^{\rm BC}_{A_{\rm O}})}{N\rho_{A_{\rm O}}}=\frac{e_{2}(1-e_{2})}{N\rho_{A_{\rm O}}}\quad(=:\rho_{A_{\rm O}}^{-1}s^{2}).$		(7)

In (6), $f_{A_{\rm O}}^{\rm C}$ represents a map from the recipient’s goodness in the present round to the donor’s goodness in the next round. Because this map is applied only when the donor cooperates, we call it “C-map”.

D-map case: On the other hand, we consider a case that the donor defects with the recipient, occurring with a probability of

$\displaystyle 1-h(p_{i_{\rm R}A_{\rm D}})=(1-p_{i_{\rm R}A_{\rm D}})(1-e_{1})+p_{i_{\rm R}A_{\rm D}}e_{1}.$

(8)

In this case, $N\rho_{A_{\rm O}}p_{i_{\rm D}A_{\rm O}}^{\prime}$ follows a probability distribution of

$\displaystyle N\rho_{A_{\rm O}}p_{i_{\rm D}A_{\rm O}}^{\prime}\sim{\cal B}(N\rho_{A_{\rm O}}p_{i_{\rm R}A_{\rm O}},a^{\rm GD}_{A_{\rm O}})+{\cal B}(N\rho_{A_{\rm O}}(1-p_{i_{\rm R}A_{\rm O}}),a^{\rm BD}_{A_{\rm O}}).$

(9)

From this equation, the mean and variance of $p_{i_{\rm D}A_{\rm O}}^{\prime}$ are given by

	$\displaystyle\textrm{E}[p_{i_{\rm D}A_{\rm O}}^{\prime}]$	$\displaystyle=p_{i_{\rm R}A_{\rm O}}(\underbrace{a^{\rm GD}_{A_{\rm O}}-a^{\rm BD}_{A_{\rm O}}}_{=:\Delta f_{A_{\rm O}}^{\rm D}})+a^{\rm BD}_{A_{\rm O}}\quad(=:f_{A_{\rm O}}^{\rm D}(p_{i_{\rm R}A_{\rm O}})),$		(10)
	$\displaystyle\textrm{Var}[p_{i_{\rm D}A_{\rm O}}^{\prime}]$	$\displaystyle=\frac{p_{i_{\rm R}A_{\rm O}}a^{\rm GD}_{A_{\rm O}}(1-a^{\rm GD}_{A_{\rm O}})+(1-p_{i_{\rm R}A_{\rm O}})a^{\rm BD}_{A_{\rm O}}(1-a^{\rm BD}_{A_{\rm O}})}{N\rho_{A_{\rm O}}}=\frac{e_{2}(1-e_{2})}{N\rho_{A_{\rm O}}}\quad(=:\rho_{A_{\rm O}}^{-1}s^{2}).$		(11)

Because the map $f_{A_{\rm O}}^{\rm D}$ is applied when the donor defects, we call it D-map in the same way as C-map.

The above C-map $f_{S_{k}}^{\rm C}$ and D-map $f_{S_{k}}^{\rm D}$ are illustrated in Fig. 1 for all $S_{k}\in{\cal S}$.

Because the population of wild-types and mutants are sufficiently large, we can continualize the distribution of individual goodness $p_{iA}$ with separating into the cases that the norm of individual $i$ is ${\rm W}$ or ${\rm M}$. In the following, $p_{AA^{\prime}}$ denotes a continualized goodness of an individual with norm $A$ in the eyes of individuals with norm $A^{\prime}$. Let us consider a time change of the distribution of $p_{AA^{\prime}}$. As shown above, however, we should keep in mind that $p_{A{\rm W}}$ and $p_{A{\rm M}}$ are simultaneously changed by the C-map or D-map. Thus, we consider dynamics of $\Phi_{A}(p_{A{\rm W}},p_{A{\rm M}})$, a joint probability distribution of $p_{A{\rm W}}$ and $p_{A{\rm M}}$. Note that a norm of the chosen recipient is ${\rm M}$ only with a probability of $\delta$, which contributes the dynamics of $\Phi_{A}$ only in a scale of $O(\delta)$. By ignoring this scale of $O(\delta)$, the dynamics of $\Phi_{A}$ is given by

	$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}\Phi_{A}(p_{A{\rm W}},p_{A{\rm M}})=-\Phi_{A}(p_{A{\rm W}},p_{A{\rm M}})+\int_{0}^{1}\int_{0}^{1}$	$\displaystyle\{h(p^{\prime}_{{\rm W}A})g(p_{A{\rm W}};f_{\rm W}^{\rm C}(p^{\prime}_{\rm WW}),\rho_{\rm W}^{-1}s^{2})g(p_{A{\rm M}};f_{\rm M}^{\rm C}(p^{\prime}_{\rm WM}),\rho_{\rm M}^{-1}s^{2})$
		$\displaystyle+(1-h(p^{\prime}_{{\rm W}A}))g(p_{A{\rm W}};f_{\rm W}^{\rm D}(p^{\prime}_{\rm WW}),\rho_{\rm W}^{-1}s^{2})g(p_{A{\rm M}};f_{\rm M}^{\rm D}(p^{\prime}_{\rm WM}),\rho_{\rm M}^{-1}s^{2})\}$
		$\displaystyle\times\Phi_{\rm W}(p^{\prime}_{\rm WW},p^{\prime}_{\rm WM})\mathrm{d}p^{\prime}_{\rm WW}\mathrm{d}p^{\prime}_{\rm WM}.$		(12)

Here, $g(p;\mu,\sigma^{2})$ denotes a Gaussian function with the mean $\mu$ and variance $\sigma^{2}$ as

$\displaystyle g(p;\mu,\sigma^{2}):=\frac{1}{\sqrt{2\pi\sigma^{2}}}\exp\left(-\frac{(p-\mu)^{2}}{2\sigma^{2}}\right).$

(13)

Equation (12) explains an update of the donor’s goodness per time. The first (resp. second) term in the right side represents decrements (increments) by updating goodnesses. In detail, $\Phi_{\rm W}(p^{\prime}_{\mathrm{WW}},p^{\prime}_{\mathrm{WM}})$ in the second term shows the density that the recipient’s goodness is $p^{\prime}_{\mathrm{WW}}$ (resp. $p^{\prime}_{\mathrm{WM}}$) in the eyes of wild-types (resp. mutants). $h(p^{\prime}_{{\rm W}A})$ shows the probability that the donor cooperates, and the donor’s goodnesses after the update in the eyes of observers with norm ${\rm W}$ and ${\rm M}$ are described by $g(p_{A{\rm W}};f_{\rm W}^{\rm C}(p^{\prime}_{\mathrm{WW}}),\rho_{\rm W}^{-1}s^{2})$ and $g(p_{A{\rm M}};f_{\rm M}^{\rm C}(p^{\prime}_{\mathrm{WM}}),\rho_{\rm M}^{-1}s^{2})$, respectively. A similar explanation holds when the donor chooses to defect.

The equilibrium state of (12), i.e., $\Phi_{A}^{*}$, satisfies

	$\displaystyle\Phi_{A}^{*}(p_{A{\rm W}},p_{A{\rm M}})=\int_{0}^{1}\int_{0}^{1}$	$\displaystyle\{h(p^{\prime}_{{\rm W}A})g(p_{A{\rm W}};f_{\rm W}^{\rm C}(p^{\prime}_{\rm WW}),\rho_{\rm W}^{-1}s^{2})g(p_{A{\rm M}};f_{\rm M}^{\rm C}(p^{\prime}_{\rm WM}),\rho_{\rm M}^{-1}s^{2})$
		$\displaystyle+(1-h(p^{\prime}_{{\rm W}A}))g(p_{A{\rm W}};f_{\rm W}^{\rm D}(p^{\prime}_{\rm WW}),\rho_{\rm W}^{-1}s^{2})g(p_{A{\rm M}};f_{\rm M}^{\rm D}(p^{\prime}_{\rm WM}),\rho_{\rm M}^{-1}s^{2})\}$
		$\displaystyle\times\Phi_{\rm W}^{*}(p^{\prime}_{\rm WW},p^{\prime}_{\rm WM})\mathrm{d}p^{\prime}_{\rm WW}\mathrm{d}p^{\prime}_{\rm WM}.$		(14)

To solve this equation, we assume that the equilibrium state can be described by a summation of two-dimensional Gaussian functions without correlation as

$\displaystyle\Phi_{A}^{*}(p_{A{\rm W}},p_{A{\rm M}})=\sum_{j}q_{Aj}g(p_{A{\rm W}};\mu_{A{\rm W}j},\rho_{\rm W}^{-1}\sigma^{2}_{A{\rm W}j})g(p_{A{\rm M}};\mu_{A{\rm M}j},\rho_{\rm M}^{-1}\sigma^{2}_{A{\rm M}j}).$

(15)

The assumption of Gaussian is justified by the above transition process of the donor’s goodness, where the goodness is virtually determined only by the mean and variance in a sufficiently large population. No correlation is assumed because the variance is given independently by observers with different norms.

We now derive equations which the equilibrium state satisfies for each norm $A\in\{{\rm W},{\rm M}\}$. First, substituting (15) into (14) for $A={\rm W}$, we obtain

	$\displaystyle\sum_{j}q_{{\rm W}j}g(p_{\mathrm{WW}};\mu_{\mathrm{WW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{WW}j})g(p_{\mathrm{WM}};\mu_{\mathrm{WM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{WM}j})$
	$\displaystyle=\int_{0}^{1}\int_{0}^{1}\{h(p^{\prime}_{\mathrm{WW}})g(p_{\mathrm{WW}};f_{\rm W}^{\rm C}(p^{\prime}_{\mathrm{WW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{WM}};f_{\rm M}^{\rm C}(p^{\prime}_{\mathrm{WM}}),\rho_{\rm M}^{-1}s^{2})$
	$\displaystyle\hskip 28.45274pt+(1-h(p^{\prime}_{\mathrm{WW}}))g(p_{\mathrm{WW}};f_{\rm W}^{\rm D}(p^{\prime}_{\mathrm{WW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{WM}};f_{\rm M}^{\rm D}(p^{\prime}_{\mathrm{WM}}),\rho_{\rm M}^{-1}s^{2})\}$
	$\displaystyle\hskip 28.45274pt\times\sum_{j}q_{{\rm W}j}g(p^{\prime}_{\mathrm{WW}};\mu_{\mathrm{WW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{WW}j})g(p^{\prime}_{\mathrm{WM}};\mu_{\mathrm{WM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{WM}j})\mathrm{d}p^{\prime}_{\mathrm{WW}}\mathrm{d}p^{\prime}_{\mathrm{WM}},$
	$\displaystyle=\sum_{j}q_{{\rm W}j}\int_{0}^{1}\int_{0}^{1}\{h(p^{\prime}_{\mathrm{WW}})g(p_{\mathrm{WW}};f_{\rm W}^{\rm C}(p^{\prime}_{\mathrm{WW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{WM}};f_{\rm M}^{\rm C}(p^{\prime}_{\mathrm{WM}}),\rho_{\rm M}^{-1}s^{2})$
	$\displaystyle\hskip 28.45274pt+(1-h(p^{\prime}_{\mathrm{WW}}))g(p_{\mathrm{WW}};f_{\rm W}^{\rm D}(p^{\prime}_{\mathrm{WW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{WM}};f_{\rm M}^{\rm D}(p^{\prime}_{\mathrm{WM}}),\rho_{\rm M}^{-1}s^{2})\}$
	$\displaystyle\hskip 28.45274pt\times g(p^{\prime}_{\mathrm{WW}};\mu_{\mathrm{WW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{WW}j})g(p^{\prime}_{\mathrm{WM}};\mu_{\mathrm{WM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{WM}j})\mathrm{d}p^{\prime}_{\mathrm{WW}}\mathrm{d}p^{\prime}_{\mathrm{WM}},$
	$\displaystyle\simeq\sum_{j}q_{{\rm W}j}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\{h(\mu_{\mathrm{WW}j})g(p_{\mathrm{WW}};f_{\rm W}^{\rm C}(p^{\prime}_{\mathrm{WW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{WM}};f_{\rm M}^{\rm C}(p^{\prime}_{\mathrm{WM}}),\rho_{\rm M}^{-1}s^{2})$
	$\displaystyle\hskip 28.45274pt+(1-h(\mu_{\mathrm{WW}j}))g(p_{\mathrm{WW}};f_{\rm W}^{\rm D}(p^{\prime}_{\mathrm{WW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{WM}};f_{\rm M}^{\rm D}(p^{\prime}_{\mathrm{WM}}),\rho_{\rm M}^{-1}s^{2})\}$
	$\displaystyle\hskip 28.45274pt\times g(p^{\prime}_{\mathrm{WW}};\mu_{\mathrm{WW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{WW}j})g(p^{\prime}_{\mathrm{WM}};\mu_{\mathrm{WM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{WM}j})\mathrm{d}p^{\prime}_{\mathrm{WW}}\mathrm{d}p^{\prime}_{\mathrm{WM}},$
	$\displaystyle=\sum_{j}q_{{\rm W}j}\{h(\mu_{\mathrm{WW}j})g(p_{\mathrm{WW}};f_{\rm W}^{\rm C}(\mu_{\mathrm{WW}j}),\rho_{\rm W}^{-1}(s^{2}+(\Delta f_{\rm W}^{\rm C})^{2}\sigma_{\mathrm{WW}j}^{2}))g(p_{\mathrm{WM}};f_{\rm M}^{\rm C}(\mu_{\mathrm{WM}j}),\rho_{\rm M}^{-1}(s^{2}+(\Delta f_{\rm M}^{\rm C})^{2}\sigma_{\mathrm{WM}j}^{2}))\}$
	$\displaystyle\hskip 28.45274pt+(1-h(\mu_{\mathrm{WW}j}))g(p_{\mathrm{WW}};f_{\rm W}^{\rm D}(\mu_{\mathrm{WW}j}),\rho_{\rm W}^{-1}(s^{2}+(\Delta f_{\rm W}^{\rm D})^{2}\sigma_{\mathrm{WW}j}^{2}))g(p_{\mathrm{WM}};f_{\rm M}^{\rm D}(\mu_{\mathrm{WM}j}),\rho_{\rm M}^{-1}(s^{2}+(\Delta f_{\rm M}^{\rm D})^{2}\sigma_{\mathrm{WM}j}^{2}))\}.$		(16)

This equation gives a constraint for $(q_{{\rm W}j},\mu_{\mathrm{WW}j},\sigma_{\mathrm{WW}j}^{2},\mu_{\mathrm{WM}j},\sigma_{\mathrm{WM}j}^{2})$. Next, when $A={\rm M}$, in a similar manner, we obtain

	$\displaystyle\sum_{j}q_{{\rm M}j}g(p_{\mathrm{MW}};\mu_{\mathrm{MW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{MW}j})g(p_{\mathrm{MM}};\mu_{\mathrm{MM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{MM}j})$
	$\displaystyle=\int_{0}^{1}\int_{0}^{1}\{h(p^{\prime}_{\mathrm{WM}})g(p_{\mathrm{MW}};f_{\rm W}^{\rm C}(p^{\prime}_{\mathrm{MW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{MM}};f_{\rm M}^{\rm C}(p^{\prime}_{\mathrm{MM}}),\rho_{\rm M}^{-1}s^{2})$
	$\displaystyle\hskip 28.45274pt+(1-h(p^{\prime}_{\mathrm{WM}}))g(p_{\mathrm{MW}};f_{\rm W}^{\rm D}(p^{\prime}_{\mathrm{MW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{MM}};f_{\rm M}^{\rm D}(p^{\prime}_{\mathrm{MM}}),\rho_{\rm M}^{-1}s^{2})\}$
	$\displaystyle\hskip 28.45274pt\times\sum_{j}q_{{\rm M}j}g(p^{\prime}_{\mathrm{MW}};\mu_{\mathrm{MW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{MW}j})g(p^{\prime}_{\mathrm{MM}};\mu_{\mathrm{MM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{MM}j})\mathrm{d}p^{\prime}_{\mathrm{MW}}\mathrm{d}p^{\prime}_{\mathrm{MM}}$
	$\displaystyle=\sum_{j}q_{{\rm M}j}\int_{0}^{1}\int_{0}^{1}\{h(p^{\prime}_{\mathrm{WM}})g(p_{\mathrm{MW}};f_{\rm W}^{\rm C}(p^{\prime}_{\mathrm{MW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{MM}};f_{\rm M}^{\rm C}(p^{\prime}_{\mathrm{MM}}),\rho_{\rm M}^{-1}s^{2})$
	$\displaystyle\hskip 28.45274pt+(1-h(p^{\prime}_{\mathrm{WM}}))g(p_{\mathrm{MW}};f_{\rm W}^{\rm D}(p^{\prime}_{\mathrm{MW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{MM}};f_{\rm M}^{\rm D}(p^{\prime}_{\mathrm{MM}}),\rho_{\rm M}^{-1}s^{2})\}$
	$\displaystyle\hskip 28.45274pt\times\sum_{j}q_{{\rm M}j}g(p^{\prime}_{\mathrm{MW}};\mu_{\mathrm{MW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{MW}j})g(p^{\prime}_{\mathrm{MM}};\mu_{\mathrm{MM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{MM}j})\mathrm{d}p^{\prime}_{\mathrm{MW}}\mathrm{d}p^{\prime}_{\mathrm{MM}}$
	$\displaystyle\simeq\sum_{j}q_{{\rm M}j}\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}\{h(\mu_{\mathrm{WM}j})g(p_{\mathrm{MW}};f_{\rm W}^{\rm C}(p^{\prime}_{\mathrm{MW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{MM}};f_{\rm M}^{\rm C}(p^{\prime}_{\mathrm{MM}}),\rho_{\rm M}^{-1}s^{2})$
	$\displaystyle\hskip 28.45274pt+(1-h(\mu_{\mathrm{WM}j}))g(p_{\mathrm{MW}};f_{\rm W}^{\rm D}(p^{\prime}_{\mathrm{MW}}),\rho_{\rm W}^{-1}s^{2})g(p_{\mathrm{MM}};f_{\rm M}^{\rm D}(p^{\prime}_{\mathrm{MM}}),\rho_{\rm M}^{-1}s^{2})\}$
	$\displaystyle\hskip 28.45274pt\times\sum_{j}q_{{\rm M}j}g(p^{\prime}_{\mathrm{MW}};\mu_{\mathrm{MW}j},\rho_{\rm W}^{-1}\sigma^{2}_{\mathrm{MW}j})g(p^{\prime}_{\mathrm{MM}};\mu_{\mathrm{MM}j},\rho_{\rm M}^{-1}\sigma^{2}_{\mathrm{MM}j})\mathrm{d}p^{\prime}_{\mathrm{MW}}\mathrm{d}p^{\prime}_{\mathrm{MM}}$
	$\displaystyle=\sum_{j}q_{{\rm M}j}\{h(\mu_{\mathrm{WM}j})g(p_{\mathrm{MW}};f_{\rm W}^{\rm C}(\mu_{\mathrm{MW}j}),\rho_{\rm W}^{-1}(s^{2}+(\Delta f_{\rm W}^{\rm C})^{2}\sigma_{\mathrm{MW}j}^{2}))g(p_{\mathrm{MM}};f_{\rm M}^{\rm C}(\mu_{\mathrm{MM}j}),\rho_{\rm M}^{-1}(s^{2}+(\Delta f_{\rm M}^{\rm C})^{2}\sigma_{\mathrm{MM}j}^{2}))\}$
	$\displaystyle\hskip 28.45274pt+(1-h(\mu_{\mathrm{WM}j}))g(p_{\mathrm{MW}};f_{\rm W}^{\rm D}(\mu_{\mathrm{MW}j}),\rho_{\rm W}^{-1}(s^{2}+(\Delta f_{\rm W}^{\rm D})^{2}\sigma_{\mathrm{MW}j}^{2}))g(p_{\mathrm{MM}};f_{\rm M}^{\rm D}(\mu_{\mathrm{MM}j}),\rho_{\rm M}^{-1}(s^{2}+(\Delta f_{\rm M}^{\rm D})^{2}\sigma_{\mathrm{MM}j}^{2}))\}.$		(17)

This equation gives a constraint for $(q_{{\rm M}j},\mu_{\mathrm{MW}j},\sigma_{\mathrm{MW}j}^{2},\mu_{\mathrm{MM}j},\sigma_{\mathrm{MM}j}^{2})$.

To solve (16), let us consider a set of solutions $\{(\mu_{\mathrm{WW}j},\mu_{\mathrm{WM}j})\}_{j}$. From the equilibrium condition of (16), the equal set must be restored by applying C-map and D-map to all the elements of the set. In other words, the condition is given by

$\displaystyle\{(\mu_{\mathrm{WW}j},\mu_{\mathrm{WM}j})\}_{j}=\{(f_{\rm W}^{\rm C}(\mu_{\mathrm{WW}j}),f_{\rm M}^{\rm C}(\mu_{\mathrm{WM}j}))\}_{j}\cup\{(f_{\rm W}^{\rm D}(\mu_{\mathrm{WW}j}),f_{\rm M}^{\rm D}(\mu_{\mathrm{WM}j}))\}_{j}.$

(18)

Similarly, to obtain the equilibrium condition for (17), we should consider a set of solutions $\{(\mu_{\mathrm{MW}j},\mu_{\mathrm{MM}j})\}_{j}$ satisfying

$\displaystyle\{(\mu_{\mathrm{MW}j},\mu_{\mathrm{MM}j})\}_{j}=\{(f_{\rm W}^{\rm C}(\mu_{\mathrm{MW}j}),f_{\rm M}^{\rm C}(\mu_{\mathrm{MM}j}))\}_{j}\cup\{(f_{\rm W}^{\rm D}(\mu_{\mathrm{MW}j}),f_{\rm M}^{\rm D}(\mu_{\mathrm{MM}j}))\}_{j}.$

(19)

Here, although the appearance of the variables are different, the problems are essentially between (18) and (19). Thus, the problem to be solved is

$\displaystyle\{(\mu_{j,{\rm W}},\mu_{j,{\rm M}})\}_{j}=\{(f_{\rm W}^{\rm C}(\mu_{j,{\rm W}}),f_{\rm M}^{\rm C}(\mu_{j,{\rm M}}))\}_{j}\cup\{(f_{\rm W}^{\rm D}(\mu_{j,{\rm W}}),f_{\rm M}^{\rm D}(\mu_{j,{\rm M}}))\}_{j}.$

(20)

Furthermore, because ${\rm W},{\rm M}\in{\cal S}$, we can generalize the problem as

$\displaystyle\{(\mu_{j,S_{01}},\cdots,\mu_{j,S_{16}})\}_{j}=\{(f_{S_{01}}^{\rm C}(\mu_{j,S_{01}}),\cdots,f_{S_{16}}^{\rm C}(\mu_{j,S_{16}}))\}_{j}\cup\{(f_{S_{01}}^{\rm D}(\mu_{j,S_{01}}),\cdots,f_{S_{16}}^{\rm D}(\mu_{j,S_{16}}))\}_{j}.$

(21)

Now, for all $S_{k}\in{\cal S}$ and, let us consider a set $\{\mu_{j,S_{k}}\}_{j\in\mathbb{Z}\backslash\{0\}}$ satisfying

	$\displaystyle\mu_{+1,S_{k}}$	$\displaystyle=f_{S_{k}}^{\rm C}(\mu_{-1,S_{k}})=f_{S_{k}}^{\rm C}(\mu_{-2,S_{k}})=\cdots,$		(22)
	$\displaystyle\mu_{+(j+1),S_{k}}$	$\displaystyle=f_{S_{k}}^{\rm C}(\mu_{+j,S_{k}}),$		(23)
	$\displaystyle\mu_{-1,S_{k}}$	$\displaystyle=f_{S_{k}}^{\rm D}(\mu_{+1,S_{k}})=f_{S_{k}}^{\rm D}(\mu_{+2,S_{k}})=\cdots,$		(24)
	$\displaystyle\mu_{-(j+1),S_{k}}$	$\displaystyle=f_{S_{k}}^{\rm D}(\mu_{-j,S_{k}}),$		(25)

(the proof for these equations will be given later). This set $\{\mu_{j,S_{k}}\}_{j\in\mathbb{Z}\backslash\{0\}}$ gives a solution to problem (21), and thus solves (16) and (17). In (22)-(25), we consistently label each $\mu_{j,S_{k}}$ such that sequentially applying C-map (resp. D-map) $j(>0)$ times leads to label $+j$ (resp. $-j$) (see the illustration in Fig. 1). In the following, we show that such $\{\mu_{j,S_{k}}\}_{j\in\mathbb{Z}\backslash\{0\}}$ actually exists for all $k$.