Peer Effects with Miss-specified Peer Groups

The literature on sampled networks, mismeasured networks and missing data has recently attracted much interest. Lewbel et al. (2019) and Hardy et al. (2019) consider the implications of measurement error in the network through which peer effects operate. Lewbel et al. (2019) also show that peer effects can be identified and consistently estimated when there is no information on the network beyond group identifiers. Boucher and Houndetoungan (2020) study peer effects when the network is unknown but that its distribution is known or can be consistently estimated.

Chandrasekhar and Lewis (2011) studies the implications of using sampled network data in applied work, provide analytical corrections in some examples of interest and develop a more general graphical reconstruction approach. In the context of peer effects, the analytical correction can be applied if the researcher has data on the network, outcomes and exogenous characteristics for a subsample of individuals and knows the identities, outcomes and exogenous characteristics of all of their peers. This means that data are only missing for peers-of-peers. To apply graphical reconstruction, for every individual, the researcher must observe the outcomes, exogenous characteristics and variables which can be used to predict the propensity of individuals to form links in a network formation model.

Breza et al. (2020) use a parametric network formation model to show that network structure can be identified and used to construct network statistics (e.g., centrality measures) if the researcher does not collect network data for all individuals, but instead collects relational data for a sub-sample of individuals and basic characteristics of all individuals.²²2Relational data are collected by asking questions of the form ‘Consider all the individuals in your group with whom you do activity X. How many of these have trait Y?’ (Breza et al., 2020) Sojourner (2013), Wang and Lee (2013) and Liu et al. (2017) consider the case in which the network and all individuals are observed without error but there are missing data in the outcomes and/or exogenous characteristics.

All of the above papers suppose that the researcher observes at least some data on every individual. In contrast, we allow for some individuals to be entirely missing and do not require knowledge on whether there even exist such individuals. These gains come from exploiting the structure of the group interactions model we consider, which, though widely used in practice (e.g., Moffitt et al. (2001); Sacerdote (2001); Glaeser et al. (2003); Angrist (2014); Boucher et al. (2014); Cornelissen et al. (2017)) and well understood theoretically (e.g., Lee (2007); Bramoullé et al. (2009); Davezies et al. (2009); Bramoullé et al. (2020)), is less general than some.³³3We discuss this issue further in Section 8.

Our work is also related to the literature on multiple peer effects. Goldsmith-Pinkham and Imbens (2013), Arduini et al. (2020) and Reza et al. (2021) study a setting in which individuals are exposed to multiple peer effects, each operating through a different network. In contrast, we suppose that individuals are exposed to peer effects operating through one network, but there is uncertainty as to which is the relevant network. Our model is thus not a model of multiple peer effects, but one of peer effects with uncertain peer groups. The approaches are not observationally equivalent. With multiple peer effects, the endogenous peer effects propogate through a network formed by taking the union of the links over multiple networks. In contrast, with uncertain peer groups, the endogenous effects propogate through only one network, the identity of which is unknown.

Our empirical application contributes to the literature on peer effects in competitive environments, particularly with respect to peers’ ability impact on lower- and medium-achieving individuals. One strand of this literature focuses on sports competitions and tournaments and mostly find a negative impact. Brown (2011), Smith (2013) and Emerson and Hill (2018) find a negative peer ability effect, in contrast to Guryan et al. (2009) who find no effect. Yamane and Hayashi (2015) find positive effects when subjects are ahead and negative effects when they are behind. Another strand of this literature focuses on educational outcomes and reports mixed findings. Some studies find significant peer effects (for example, Hoxby (2000) and Sacerdote (2001) find a positive effect and Antecol et al. (2016) find negative effect), and others find small to no effects (for example, Foster (2006) and Angrist and Lang (2004)). Brady et al. (2017) find negative effects at broader level but positive effects at smaller level, suggesting that peer effects operate differently for different groups. The overall perception is that effects are not identical across different ability groups. Low-achieving groups are adversely affected by peer ability and may benefit from tracking compared to ability-mixing groups (duflo2011pee; Carrell et al., 2013; Booij et al., 2017). Our work builds on these papers by examining career decisions in the highly competitive and incentivized labor market for junior lawyers. Our results are more in line with the literature finding peer ability hurts subjects’ persistence in education and in their chosen career Thiemann (2022); Howell (2021).

We first consider identification when there are missing data. Our analysis makes uses of the indicator $s_{i}\in\{0,1\}$ for the condition $i\in\mathcal{S}$.

Assumption 2 covers the simplest case in which the group size is known but individuals are sampled from the group. In practice, it means that the researcher has access to a sample of individuals with outcomes, characteristics, group membership indicators and group size. The distribution of $s_{i}$ (i.e., the inclusion probability) can be heterogeneous across individuals, but ought not to depend on the structural error (i.e., there should be no sample selection on unobservables). Identification under Assumption 2 was first considered by Davezies et al. (2009). We include it for comparison with our approach, which allows the group size to be unknown.

Assumption 3 relaxes the requirement that the researcher knows the group size. In practice, it allows for the case in which the researcher observes an individual level sample of outcomes, exogenous characteristics and group membership indicators. In such a sample, the researcher knows the number of individuals sampled from each group ($n_{g}$) but does not know the group size ($n_{g_{0}}$), nor whether there are any missing individuals in each group. Relative to Assumption 2, we additionally assume that the sample of observed individuals is unbiased and uncorrelated. These are mild assumptions likely to be satisfied in well designed individual level samples under common sampling schemes (Arnab, 2017).

Assumptions 2 and 3 both maintain that the sampling of individuals does not induce sample selection, whilst Assumption 3 additionally maintains that the sampling does not depend on exogenous covariates nor on the group size. Viewed from the missing data perspective, Assumption 3 is essentially a type of Missing Completely at Random assumption, which is widely used (often implicitly) in empirical work. Together, $\mathbb{E}[s_{i}|\mathbf{x}_{g_{0}},n_{g_{0}}]=\rho\in(0,1]$ and $\mathbb{COV}[s_{i},s_{j}|\mathbf{x}_{g_{0}},n_{g_{0}}]=0$ imply

which leads to identification of the distribution $n_{g_{0}}|n_{g},\mathbf{x}_{g}$ from the observed distribution $n_{g}|n_{g}\geq 1,\mathbf{x}_{g}$.¹⁰¹⁰10Conditioning on $n_{g}\geq 1$ is important, because for some groups the researcher may not observe any individuals. Hence it is the distribution $n_{g}|n_{g}\geq 1,\mathbf{x}_{g}$ which is observed rather than $n_{g}|\mathbf{x}_{g}$.

We now briefly discuss possible generalizations of Assumption 3, and the extent to which they facilitate identification. First, one might consider $\rho=\rho_{g_{0}}$ to allow for group heterogeneity. Such an assumption does not provide information to identify $n_{g_{0}}|n_{g},\mathbf{x}_{g}$ because $\mathbb{P}[n_{g_{0}}=m|\mathbf{x}_{g}]$ and $\rho_{g_{0}}$ only appear multiplicatively in the expression for $\mathbb{P}[n_{g}=n|n_{g}\geq 1,\mathbf{x}_{g}]$ for all $m$ and $n$. For the same reason, using $\rho=\rho(n_{g_{0}})$ does not provide identifying information.

Alternatively one might consider $\rho=\rho(z_{i})$ for some exogenous observable $z_{i}$ (e.g., $z_{i}=x_{i}$). Viewed from the missing data perspective, this corresponds to a Missing At Random assumption. In this case $n_{g}|n_{g_{0}},\mathbf{x}_{g_{0}},\mathbf{z}_{g_{0}}$ follows a Poisson Binomial distribution, which could, in principle, be used for identification. However, if $\mathbf{z}_{g_{0}}$ is observed then $n_{g_{0}}$ is known and identification is attained instead under the weaker Assumption 2. In contrast, the distribution $n_{g}|n_{g_{0}},\mathbf{x}_{g},\mathbf{z}_{g}$ does not take a known form, hence is of limited use for the identification in the typical setting in which we have no information on the missing individuals (i.e., there are some individuals for whom $z_{i}$ is unobserved).

We do not pursue the above generalisations formally because the focus of our analysis is not on sample selection at the individual level, but instead on replacing a group level sample with an individual level sample. Hence we consider Assumption 3 as the benchmark case in which the researcher has access to a well designed sample of individuals rather than a well designed sample of groups. We now present our identification result.

Result 1 was first established by Davezies et al. (2009) for missing data with known group sizes whereas result 2 applies when the group sizes are unknown. Point identification of $\rho$ comes in part from information on the lower bound of the support of $n_{g_{0}}$. The result uses that $n_{g_{0}}\geq 2$, which is maintained throughout the theoretical and empirical literature (e.g., Lee (2007); Bramoullé et al. (2009); Davezies et al. (2009); Bramoullé et al. (2020); Moffitt et al. (2001); Sacerdote (2001); Glaeser et al. (2003); Angrist (2014); Boucher et al. (2014); Cornelissen et al. (2017)). Indeed, groups of size one cannot provide information on peer effects and are not consistent with our model. However, if individuals are missing, then we may observe groups with only one member even though all groups have at least two members (i.e., $\mathbb{P}[n_{g}=1|\mathbf{x}_{g},n_{g}\geq 1]>0$ if $\rho<1$). Hence the distribution of $n_{g}$ provides information on $\rho$, which combined with (4.4), leads to identification of $\rho$ and the distribution $n_{g_{0}}|n_{g},\mathbf{x}_{g}$. Equation (4.3) can then be used to identify the peer effects.

In some applications a stronger support restriction may be used if it is known that $n_{g_{0}}\geq\underline{n}$ for some known $\underline{n}>2$. For example, this is likely to be satisfied in studies of peer effects in education, where classrooms might reasonably be assumed to contain at least a handful of students. Though not required for identification, this can improve the precision of the GMM estimator we propose below. In other applications $n_{g_{0}}=1$ may be feasible, in which case one can never rule out $\rho=1$ and $n_{g_{0}}=n_{g}$ so that peer effects are not identifiable. For this reason, a restriction on the lower bound of the support of $n_{g_{0}}$ is necessary for identification of the peer effects.

Result 2 also uses the mild assumption that the distribution of $n_{g_{0}}$ is bounded, which is needed to identify $\rho$ and the distribution of $n_{g_{0}}|\mathbf{x}_{g}$ from the distribution of $n_{g}|n_{g}\geq 1,\mathbf{x}_{g}$ as a solution of a non-linear system comprising a finite number of equations. The upper bound need not be known since it is identifiable from the distribution of $n_{g}|n_{g}\geq 1$. This assumption is also used, for example, by Lee (2007), Davezies et al. (2009), Lewbel (2019) and Boucher and Houndetoungan (2020).

We now consider subset miss-specification resulting from group uncertainty.

Assumption 4 covers the case in which there is group uncertainty but the candidate groups are naturally nested. Though an extension to three or more is straightforward, for simplicity of exposition we do not pursue it formally. We also require a minor strengthening of the moment condition, replacing (2.2) with $\mathbb{E}[\epsilon_{i}|n_{g_{1}},n_{g_{2}},\mathbf{x}_{g_{2}}]=0$, and hence $n_{g}$ with $n_{g_{1}},n_{g_{2}}$ in equations (4.2) and (4.3). The stronger moment condition requires that both potential group sizes and the exogenous characteristic of all individuals in the two potential groups be strictly exogenous with respect to the structural error.

Under Assumption 4, the reduced form is

where $\varphi^{\prime}(n_{1},n_{2},\mathbf{x})\triangleq\mathbb{E}[\pi(n_{g_{0}})|n_{g_{1}}=n_{1},n_{g_{2}}=n_{2},\mathbf{x}_{g_{2}}=\mathbf{x}]=\psi\pi(n_{1})+(1-\psi)\pi(n_{2})$, hence identification hinges on the properties of the joint distribution of $n_{g_{1}}$ and $n_{g_{2}}$, specifically, through the bi-partite graph of the support of $(n_{g_{1}},n_{g_{2}})$, which we denote by $\mathcal{G}_{n_{g_{1}},n_{g_{2}}}$. Figure 1 depicts an example.

Our approach can be generalized to allow $\psi=\psi(\mathbf{z}_{g_{2}})$ (e.g., $\mathbf{z}_{g_{2}}=\mathbf{x}_{g_{2}}$), provided that the conditional expectations and probabilities in Assumption 4 continue to hold conditional also on $\mathbf{z}_{g_{2}}$. We discuss the corresponding modification of Proposition 2 at the end of this section. We now present our identification result.

The intuition for identification of $\psi\pi(n_{1})+(1-\psi)\pi(n_{2})$ comes from a regression of $\overline{y}_{i}$ on the interaction of $\overline{x}_{i}$ with dummies for each value of $n_{g_{1}}$ and the interaction of $\overline{x}_{i}$ with dummies for each value of $n_{g_{2}}$. Together, both sets of dummies sum to two for every individual in the sample (because each individual is in exactly two candidate groups, a small one and a larger one), hence one dummy must be omitted. For this reason, we identify the pairwise sums of $\psi\pi(n_{1})$ and $(1-\psi)\pi(n_{2})$ for all pairs in the same connected component. If we observe a connected component containing at least three vertices corresponding to $n_{g_{1}}$ we can contrast $\psi\pi(n_{g_{1}})$ over three different values of $n_{g_{1}}$ holding $n_{g_{2}}$ constant. This yields three equations in the four unknowns $\psi,\gamma,\delta,\beta$. We obtain an additional equation by repeating the exercise for a different value of $n_{g_{2}}$, which leads to identification.

Under the generalization of Assumption 4 to $\psi=\psi(\mathbf{z}_{g_{2}})$, Proposition 2 holds provided that the support conditions on $n_{g_{1}}$ and $n_{g_{2}}$ hold conditionally on $\mathbf{z}_{g_{2}}=\mathbf{z}$ for some element $\mathbf{z}$ of the support of $\mathbf{z}_{g_{2}}$. Similarly, one replaces $\mathcal{G}_{n_{g_{1}},n_{g_{2}}}$ by $\mathcal{G}_{n_{g_{1}},n_{g_{2}}|\mathbf{z}_{g_{2}}=\mathbf{z}}$. This rules out pathological cases such as $z_{i}=n_{g_{2}(i)}$.

We consider a design satisfying Assumption 3. The original data comprise a complete sample of freshman (i.e., $\rho=1$). Our design varies $\rho\in\{0.3,0.5,0.7,0.9,1\}$. For $\rho=1$ we set $s_{i}=1$ for all $i\in\{1,...,N_{0}\}$ and for $\rho<1$ we set $p_{i}=\rho+q_{i}$, where $q_{i}\overset{i.i.d.}{\sim}\text{Uniform}(-0.1,0.1)$ and $s_{i}=1$ with probability $p_{i}$. We set $G_{0}$ to be the nearest integer to $M/(\rho\mathbb{E}[n_{g_{0}}])$, where $M$ is discussed below. For each dataset, we draw $G_{0}$ dorm-rooms from the distribution of dorm-room sizes, which we take to be $n_{g_{0}}-2\sim{\rm Binomial}(2,0.25)$ so as to match the sample mean and support of the Dartmouth data. We use a parametric distribution so as to evaluate the performance of the GMM estimator both with and without a parametric restriction on the group size distribution.

Since we draw a fixed number of dorm rooms, each with a random size, the number of students $N_{0}$ varies from one dataset to the next. We then draw an independent random sample of $N$ students, which includes student $i$ with probability $p_{i}$. Hence $N$ also varies form one dataset to the next. Allowing $G_{0}$ to depend on $\rho$ as above implies that the size of the observed sample is $N\approx M$ no matter the value of $\rho$. Setting $M=1600$, our design matches the sample size of the Dartmouth data. We thus consider the performance of our method had the Dartmouth data been obtained from a random sample of freshman, rather than all freshman, holding the number of observations constant as we vary $\rho$. We also consider $M=8000$ to study the performance of the method under different sample sizes.

Sacerdote (2001) exploits random assignment of freshmen to dorms in the Dartmouth data to identify peer effects in educational attainment, measured by freshman GPA. It is argued that endogenous effects are difficult to identify due to the reflection problem (see Manski (1993)). Sacerdote (2001) focuses instead on credibly identifying the reduced form effect of room-mate high school attainment on freshman GPA (see column 5 of Table 3 in Sacerdote (2001)). The estimated peer effect is positive and statistically significant, though smaller in magnitude than own high-school attainment. This reduced form regression has $R^{2}=0.19$.

To match this setting as well as possible, in our design we set the own effect to be $\gamma=1$ and the endogenous effect to be $\beta=0$, hence the reduced form effect of room-mate’s high-school attainment is equal to the contextual effect, which we set to be $\delta=0.5$. We set $x_{i}\overset{i.i.d.}{\sim}\mathcal{N}(0,1)$, $\epsilon_{i}\overset{i.i.d.}{\sim}\mathcal{N}(0,\sigma^{2})$ and $\alpha_{g_{0}}\overset{i.i.d.}{\sim}\mathcal{N}(1,\sigma^{2})$ and choose $\sigma^{2}=2(\gamma^{2}+\delta^{2}/\mathbb{E}[n_{g_{0}}-1])$. For the expected dorm-room size, this choice of $\sigma^{2}$ corresponds to fraction 0.8 of $\mathbb{V}[y_{i}]$ being due to $\alpha_{g_{0}(i)}+\epsilon_{i}$ and 0.2 being due to $x_{i}\gamma+(1-n_{g_{0}})^{-1}\sum_{j:g_{0}(j)=g_{0}(i),j\neq i}x_{j}$, hence a population $R^{2}$ of 0.2 in a reduced form regression without fixed effects, as estimated by Sacerdote (2001).

Sacerdote (2001), Glaeser et al. (2003) and Angrist (2014) all discuss the appropriate definition of the peer group, which may either be the room, the floor or the entire dorm. We focus here on the distinction between the room and the floor, and consider a design in which Assumption 4 holds. To match the Dartmouth data, we first draw the rooms as described above. We then draw $f_{1}\in\{1,2,...,5\}$ uniformly, and take rooms $\{1,2,...,f_{1}\}$ to comprise the first floor. We then draw another integer $f_{2}\in\{1,2,...,5\}$ and take rooms $\{f_{1}+1,...,f_{1}+f_{2}\}$ to comprise the second floor. We proceed in this way until every room has been allocated to a floor. In the Dartmouth data, the sample mean number of students per floor is close to 8. This data generating process yields an expected number of students per floor of 7.5. All other aspects of the data generating process remain unchanged and we vary $\psi\in\{0.2,0.4,0.6,0.8\}$.

Our baseline results are reported in Table 5. Models (1)-(2) respectively use group structures F and F-C. We find a positive yet statistically insignificant peer effect in fee-weighted caseload. In group F, the magnitude of the peer effect is around one third of the effect of a lawyer’s own fee-weighted caseload, which is negative and statistically significant at the 0.01 level. For group F-C, the magnitude of the peer effect is similar to that of own fee-weighted caseload. Models (3)-(4) consider instead the proportion of high-ability peers in groups F and F-C respectively, for which we find larger peer effects. The coefficient of 0.0544 in model (3) is statistically significant at the 0.05 level and indicates that a one standard deviation (17.2 pp) increase in the proportion of high-ability peers leads to 26.0 percent (0.9 pp) increase in lawyers’ quit-to-exit probability. We find a similar effect in model (4). In both models we find a notable gender gap, in that female lawyers are around 35.8-41.1 percent (1.3-1.5 pp) more likely to quit than male lawyers. This is consistent with previous findings that women are more likely to discontinue in competitive environments (Hunt, 2016; Wasserman, 2018). We also find evidence that quits are negatively but concavely correlated with own age and negatively correlated with own fee-weighted caseload, which is consistent with Jovanovic (1979).

Next, in models (5)-(6) we estimate the impact of peer ability separately for those above and below 35. We find that the peer effect is stronger in the younger cohort. The coefficient of 0.0767 in model (5) is statistically significant at the 0.01 level and indicates that a one standard deviation (19.3 pp) increase in the proportion of high-ability peers leads to 35.9 percent (1.48 pp) increase in lawyers’ quit-to-exit likelihood. In contrast, we find a smaller effect among those older than 35, which is not statistically distinguishable from zero. We also find a larger gender gap among those below 35. Overall, we find stronger quit-to-exit effects among those under 35, which is consistent with Jovanovic (1979).

In Table 6, we restrict our analysis to small- to medium- size firms to alleviate the concern that our ability measure cannot capture non-litigation performance. We limit the sample of firms to those have 50 or fewer lawyers (i.e. 732 out of 755 firms). Our results are qualitatively identical the baseline specification. Another concern is on lawyers close to the retirement age, who may intentionally reduce caseload before the retirement. To that end, we replicate our preferred models (models (3)-(4) in Table 5) and we exclude lawyers beyond age 55, the age at which most employees in China are eligible to claim their pension. As shown in models (1)-(2) of Table 7, we still find similar adverse peer ability effects, indicating that retiring lawyers are not driving the estimated effects. Next, in models (3)-(4), we conduct another robustness check by including all law firms in Shanghai. The similar effects we find in these two columns suggest that caseload broadly serves as a reliable ability proxy in the legal profession context. Overall, the similar effects found in alternative samples in models (1)-(4) and in Table 6 imply that the estimated peer ability effects generally exist in our context. Lastly, in models (5)-(6), we consider an alternative definition using the number of high-ability peers in the specific groups F and F-C. The results are also in line with our baseline findings.

In Table 8, we conduct a heterogeneity analysis. In models (1)-(2), we examine the impact of high-ability peers on lawyers whose fee-weighted caseload is below median. Consistent with Antecol et al. (2016) and Booij et al. (2017), these low- to medium-ability individuals drive the negative peer ability effects. In models (3)-(4), we study whether women are more sensitive to peer ability effects. Despite the significant gender gap of persistence, we find no evidence suggesting that women are more sensitive to peer ability effects. In models (5)-(6), we check whether the magnitude of peer ability effects vary by firm size by interacting the proportion of high-ability peers with an indicator for firms below the median size. The coefficient on the interaction term suggests that lawyers in small firms are not additively more prone to peer ability effects.

We now ask whether we could have obtained similar findings had only a random sample of lawyers with firm identifiers been available. To do this, we draw random 1000 random sub-samples of lawyers using the sampling process from our Monte-Carlo experiment (see Section 6.1). We focus on the baseline specification for group F (model (3) of Table 5), consider 50% and 70% subsamples (i.e., $\rho=0.5$ and $\rho=0.7$). The minimum value of $n_{g_{0}}$ in our sample is 2, which we use as a lower bound for the estimator with unknown group sizes. We make a parametric restriction on the distribution of $n_{g_{0}}-2$, supposing that it follows a negative binomial distribution. Figure 2 depicts the empirical distribution for the original sample and a fitted negative binomial distribution, which provides a good approximation.

The results are reported in Figures 3 ($\rho=0.5$) and 4 ($\rho=0.7$). As in our Monte-Carlo experiment, we consider the estimator based on miss-specified peer group sizes (i.e., incorrectly supposing that $n_{g}=n_{g_{0}}$), known sizes and unknown sizes. Looking at the left hand columns we see that miss-specification has little consequence for estimates of the effect of a lawyer’s own characteristics. However, looking at the right hand columns we see that miss-specification biases estimates of peer effects towards zero. Comparing Figure 3 with Figure 4, we see that the bias in in the peer effects is larger for $\rho=0.5$ than for $\rho=0.7$. Unsurprisingly, the variability of the estimators is also larger for $\rho=0.5$ than for $\rho=0.7$. These findings are qualitatively identical to those of our Monte-Carlo experiment, but use real data and larger group sizes.

Model (7) of Table 5 allows for uncertainty as to whether F or F-C is the relevant peer group. The point estimate of $\psi$ (i.e., the probability of F-C) is 0.562, which suggests considerable heterogeneity, though it is not precisely estimated. Despite this, we find similar peer effects to the models based on groups F and F-C. We obtain similar results in the sample of small-medium firms (see Table 6).