Standard Errors When a Regressor is Randomly Assigned

We start by considering a simple univariate linear regression time series model in which we have

where $(\varepsilon_{i})_{i\leq n}$ is a second-order, possibly autocorrelated, stationary time series – we employ the index $i$, rather than $t$, to emphasize our analysis is not confined to the time series context. We depart from the textbook time series model by assuming that the regressors $(d_{i})_{i\leq n}$ are: (i) Independent and identically distributed (i.i.d.) with mean zero, and (ii) Strongly exogenous in the sense that $(d_{i})_{i\leq n}$ is independent of the time series process $(\varepsilon_{i})_{i\leq n}$.

As is well-known, the least squares estimator $\hat{\beta}$ of $\beta$ in model (1) satisfies the equality

In many standard time series textbooks, the asymptotic distribution of $\hat{\beta}$ is thus derived by imposing sufficiently strong conditions to ensure that the score $n^{-1/2}\sum_{i=1}^{n}d_{i}\varepsilon_{i}$ is asymptotically normal and the Hessian $n^{-1}\sum_{i=1}^{n}d_{i}^{2}$ converges in probability to a non-stochastic matrix. In order to derive the standard error for $\hat{\beta}$, we therefore only need a consistent estimator of the long-run variance of the score; i.e., a heteroscedasticity and autocorrelation consistent (HAC) variance estimator, such as those proposed by Newey and West (1987) and Andrews (1991).

On the other hand, if the regressors $(d_{i})_{i\leq n}$ are i.i.d. and strongly exogenous with mean zero, the independence of $(d_{i})_{i\leq n}$ and $(\varepsilon_{i})_{i\leq n}$ implies that for any $1\leq i_{1},i_{2}\leq n$ with $i_{1}\neq i_{2}$, we must have

and also $\mathbb{E}[(d_{i}\varepsilon_{i})^{2}]=\mathbb{E}[d_{i}^{2}]\mathbb{E}[\varepsilon_{i}^{2}]$ for all $1\leq i\leq n$. Hence, as long as some version of the central limit theorem is applicable to the score $n^{-1/2}\sum_{i=1}^{n}d_{i}\varepsilon_{i}$ and a law of large numbers is applicable to the Hessian $n^{-1}\sum_{i=1}^{n}d_{i}^{2}$, we can conclude that the asymptotic distribution of $\hat{\beta}$ is given by

In particular, statistical inference on $\beta$ can be conducted as if it were not a time series model, i.e. using the formula $s^{2}\left(\mathbf{X}^{\top}\mathbf{X}\right)^{-1}$, where $\mathbf{X}\equiv(d_{i})_{i\leq n}$ in this case.

The main takeaway of the preceding example is that the strong exogeneity and i.i.d. nature of the regressors $(d_{i})_{i\leq n}$ imply that the sequence $(d_{i}\varepsilon_{i})_{i\leq n}$ is homoscedastic even if the errors $(\varepsilon_{i})_{i\leq n}$ are arbitrarily autocorrelated. Is this simplification confined to the time series model? Our preceding discussion suggests that this is not the case. Indeed, provided the regressors $(d_{i})_{i\leq n}$ are i.i.d., mean zero, and strongly exogenous, the score $n^{-1/2}\sum_{i=1}^{n}d_{i}\varepsilon_{i}$ has a built-in martingale structure vis-à-vis the filtration $\mathcal{F}_{i}$ generated by $(d_{j},\varepsilon_{j})_{j\leq i}^{\top}$ because:

Therefore, assuming that the random pairs $(d_{i},\varepsilon_{i})$ satisfy certain moment conditions, the martingale central limit theorem will be applicable regardless of the dependence structure of $(\varepsilon_{i})_{i\leq n}$ and the long run variance of the score will reduce to $\mathbb{E}[d_{i}^{2}]\mathbb{E}[\varepsilon_{i}^{2}]$. In particular, the variance formula $s^{2}\left(\mathbf{X}^{\top}\mathbf{X}\right)^{-1}$ will remain valid despite the dependence present in the variables $(\varepsilon_{i})_{i\leq n}$. Thus, spatial correlation, network dependence, and/or a cluster structure in the variables $(\varepsilon_{i})_{i\leq n}$ are all accommodated by the standard homoscedastic standard errors. Moreover, we note that a quick inspection at the preceding argument reveals that the assumptions we have imposed so far are stronger than necessary for the desired conclusion to hold.

We next build on our preceding discussion by considering the multivariate linear regression model

where $d_{i}$ is a scalar regressor of interest, $w_{i}$ is a $d_{w}$-vector of controls, and $(w_{i},\varepsilon_{i})_{i\leq n}$ is a second-order, possibly autocorrelated, stationary time series satisfying $\mathbb{E}[w_{i}\varepsilon_{i}]=\mathbf{0}_{d_{w}\times 1}$ for all $1\leq i\leq n$. We continue to assume that the regressors $(d_{i})_{i\leq n}$ are i.i.d. with mean zero, and strongly exogenous in the sense that $(d_{i})_{i\leq n}$ is independent of the time series process $(\varepsilon_{i},w_{i}^{\top})_{i\leq n}$. The parameter of interest continues to be $\beta$.

For this model, the Frisch-Waugh-Lovell theorem implies the least squares estimator $\hat{\beta}$ satisfies

Hence, under appropriate regularity conditions the estimator $\hat{\beta}$ admits the asymptotic expansion

In particular, if $(d_{i})_{i\leq n}$ is mean zero and independent of $(w_{i}^{\top})_{i\leq n}$, then $\alpha=0$ and the asymptotic expansion reduces to the univariate setting – i.e. the right-hand side of (6) is (asymptotically) equivalent to the right-hand side of (2). It therefore follows that the end result is the same as in the univariate case: The variance formula $s^{2}\left(\mathbf{X}^{\top}\mathbf{X}\right)^{-1}$, where $\mathbf{X}\equiv(d_{i},w_{i}^{\top})_{i\leq n}$ in this case, remains valid for $\hat{\beta}$. We emphasize, however, that this formula is not necessarily justified for conducting inference on the coefficient $\gamma$.

As a preview of results in the next section, we again note that the conditions we have imposed so far are stronger than required. For instance, suppose that instead of demanding that the regressors $(d_{i})_{i\leq n}$ and $(w_{i}^{\top})_{i\leq n}$ be fully independent, we assume that they are related according to the model

for $(\eta_{i})_{i\leq n}$ i.i.d. and independent of $(\varepsilon_{i},w_{i}^{\top})_{i\leq n}$. The asymptotic expansion in (6), combined with the same arguments employed in the univariate case, then continue to imply that

under mild moment conditions – i.e. when computing standard errors for $\hat{\beta}$ we can continue to pretend that the time series process fits the textbook homoscedastic model. Setting $w_{i}$ to be a constant, for instance, reveals that the mean zero assumption on $(d_{i})_{i\leq n}$ is superfluous.

The preceding martingale argument relies crucially on two key assumptions: (i) The regressors $(d_{i})_{i\leq n}$ are independent of each other, and (ii) The regressors $(d_{i})_{i\leq n}$ are independent of the errors $(\varepsilon_{i})_{i\leq n}$. A challenge to our martingale argument arises when $(d_{i})_{i\leq n}$ and $(\varepsilon_{i})_{i\leq n}$ are not independent. Within the potential outcome framework, for instance, this full independence requirement is violated in the presence of heterogenous treatment effects. More precisely, heterogenous treatment effects render $\varepsilon_{i}$ conditionally heteroscedastic with respect to the treatment status $d_{i}$. Motivated by this observation, we also study a model in which $\varepsilon_{i}=\sum_{l=1}^{L}\sigma_{l}(d_{i})\varepsilon_{l,i}^{*}$ with $e_{i}\equiv(\varepsilon_{l,i}^{*})_{l\leq L}$ possibly correlated across $i$ and $l$, but $(e_{i})_{i\leq n}$ fully independent of $(d_{i})_{i\leq n}$. In the potential outcome framework, with $d_{i}\in\{0,1\}$ indicating treatment status, we would have

where $y_{i}(d)$ denotes the potential outcome for unit $i$ under treatment status $d\in\{0,1\}$.

To see the problem for the martingale structure in this model, observe that for any $1\leq i_{1},i_{2}\leq n$, we now have

which is not necessarily zero even if $d_{i}$’s are mean zero. The variance formula for the sum $\sum_{i\leq n}d_{i}\varepsilon_{i}$ therefore must include interactions terms as long as the random vectors $(\varepsilon_{l,i_{1}}^{*})_{l\leq L}$ and $(\varepsilon_{l,i_{2}}^{*})_{l\leq L}$ are correlated.

We will present a detailed analysis of the conditional heteroscedasticity case in Section 3 but the main takeaways from our results are: (i) The independence of $(d_{i})_{i\leq n}$ from controls $(w_{i}^{\top})_{i\leq n}$ still simplifies the asymptotic variance for $\hat{\beta}$; and (ii) Conditional heteroscedasticity yields a break in the martingale structure that requires us to adjust standard errors not only for heteroscedasticity, but also for correlation of the errors across units. For example, in the context of (7), our analysis implies the asymptotic variance of $\sqrt{n}(\hat{\beta}-\beta)$ equals (the probability limit of)

In particular, we note that standard errors may need to be adjusted for correlation if we are concerned the treatment effects are correlated. On the other hand, the correlation between components of the vector $(y_{i}(0))_{i\leq n}$ plays no role for the standard errors, and neither does correlation between components of the vector $(y_{i}(1))_{i\leq n}$.

We first study the case that $\mathbf{D}$ is independent of $\bm{\epsilon}$, and hence $\bm{\epsilon}$ is conditionally homoscedastic with respect to $\mathbf{D}$. Together with independence of $\mathbf{D}$ from $\mathbf{W}$, this means that $\mathbf{D}$ is strongly exogenous. The conditional heteroscedasticity case is discussed later in this section. In the assumptions that follow, recall that $K$ should be interpreted to denote a sufficiently large constant that can change from place to place but is independent of the sample size $n$.

Assumption 1 contains our main requirements for the regressor of interest. In particular, Assumptions 1(i) and 1(ii) are our key conditions that are satisfied in many randomized control trials. Assumptions 1(iii) and 1(iv) are mild moment conditions. Assumption 1(v) means that $(d_{i})_{i\leq n}$ is strongly exogenous. Assumption 2 contains our main requirements for the controls. In particular, Assumption 2(i) means that there is no multicollinearity among controls. Assumption 2(ii) is a mild moment condition. Assumption 2(iii) means that we study regressions with an intercept. Assumption 3 contains our main requirements for the regression error. Assumption 3(i) holds if $\varepsilon_{i}$’s are uncorrelated with $w_{i}$’s and a law of large numbers applies to the product $w_{i}\varepsilon_{i}$. Assumption 3(ii) is essentially a law of large numbers for $\varepsilon_{i}^{2}$’s. Assumptions 3(iii) and 3(iv) are mild moment conditions. We highlight that our assumptions allow for a wide array of dependence structures in the matrix $(\varepsilon_{i},w_{i}^{\top})_{i\leq n}$, with the main condition in this regard intuitively being that dependence be “weak” enough for the law of large numbers imposed in Assumptions 3(i) and 3(ii) to apply.

For all $1\leq i\leq n$, denote $d_{i}^{*}\equiv d_{i}-\mu_{d}$. The following theorem derives the asymptotic distribution of the OLS estimator $\hat{\beta}$ in the strongly exogenous case.

This theorem establishes two key facts. First, it shows that, given the strong exogeneity of $(d_{i})_{i\leq n}$, our mild requirements on the dependence structure of $(\varepsilon_{i},w_{i}^{\top})_{i\leq n}$ suffice for establishing asymptotic normality of $\hat{\beta}$. To establish such a conclusion, we rely on a martingale construction that generalizes our discussion in Section 2. Second, Theorem 1 establishes that the asymptotic variance of $\hat{\beta}$ is not affected by the possible correlation across vectors $w_{i}\varepsilon_{i}$ since it only depends on the variance of $d_{i}$ and the averaged variance of the error terms $(\varepsilon_{i})_{i\leq n}$. We emphasize that neither of these conclusions need hold for the estimator $\hat{\gamma}$ of the coefficient $\gamma$ corresponding to the vectors of controls $w_{i}$.

In addition, Theorem 1 suggests that we can estimate the variance of $\hat{\beta}$ by $s^{2}(\mathbf{\breve{D}^{\top}\breve{D}})^{-1}$, where

$\mathbf{\breve{D}}\equiv\mathbf{M}_{W}\mathbf{D}$ and $\mathbf{M}_{W}\equiv\mathbf{I}_{n}-\mathbf{W}(\mathbf{W}^{\top}\mathbf{W})^{-1}\mathbf{W}^{\top}$. The following corollary confirms this conjecture.

Together with Theorem 1, this corollary is our first main result. Indeed, it is well-known that the top left element of the matrix $(\mathbf{X}^{\top}\mathbf{X})^{-1}$ coincides with $(\mathbf{\breve{D}}^{\top}\mathbf{\breve{D}})^{-1}$. Therefore, this corollary justifies using the classic standard error formula $s^{2}\left(\mathbf{X}^{\top}\mathbf{X}\right)^{-1}$ for inference on $\beta$, even though we allow for general dependence structures in the errors $(\varepsilon_{i})_{i\leq n}$ and controls $(w_{i}^{\top})_{i\leq n}$.

Next, we derive the asymptotic variance for $\hat{\beta}$ in the case of conditional heteroscedasticity, i.e. when $(\varepsilon_{i})_{i\leq n}$ is conditionally heteroscedastic with respect to $(d_{i})_{i\leq n}$. Following the notation introduced in Section 2, we focus on the case in which $\varepsilon_{i}\equiv\sum_{l\leq L}\sigma_{l}(d_{i})\varepsilon_{l,i}^{\ast}$, where $e_{i}\equiv(\varepsilon_{l,i}^{\ast})_{l\leq L}$ is a vector with mean zero, for all $1\leq i\leq n$.

Let $A_{i}\equiv(d_{i}-\mu_{d})(\sigma_{l}(d_{i}))_{l\leq L}$ for all $1\leq i\leq n$ and observe that under Assumption 1(i), the random vectors $A_{i}$ are i.i.d. Denote their common mean vector by $\mu_{A}$. In addition, denote $\sigma_{e,1}^{2}\equiv n^{-1}\sum_{i=1}^{n}\mathbb{E}[((A_{i}-\mu_{A})^{\top}e_{i})^{2}]$ and $\sigma_{e,2}^{2}\equiv\mathbb{E}[(n^{-1/2}\sum_{i=1}^{n}\mu_{A}^{\top}e_{i})^{2}]$. Within this context, we impose the following assumptions.

Assumption 4 is mainly used to replace Assumption 1(v) and accounts for the conditional heteroscedasticity of $(\varepsilon_{i})_{i\leq n}$ with respect to $(d_{i})_{i\leq n}$. Assumption 4(i) requires that $(d_{i})_{i\leq n}$ are strongly exogenous with respect to the“scaled” error vector $(e_{i})_{i\leq n}$ – a requirement that, as discussed in Section 2 maps well into a potential outcome framework with heterogeneous treatment effects. Assumption 4(ii) imposes upper bounds on the functions $\sigma_{l}^{2}(\cdot)$, which we view as a mild regularity condition. Assumption 4(iii) is a mild moment condition. Assumption 5 contains further restrictions on the vectors $e_{i}$. Assumption 5(i) is essentially a law of large numbers for $e_{i}e_{i}^{\top}$’s. Assumption 5(ii) is essentially a moment condition for the random variables $\|e_{i}\|$. Assumption 5(iii) limits the amount of dependence among the vectors $e_{i}$ to ensure convergence in distribution.

The following theorem, which is our second main result, derives the asymptotic distribution of the OLS estimator $\hat{\beta}$ in the case of conditional heteroscedasticity.

To establish this theorem, we decompose the score $n^{-1/2}\sum_{i=1}^{n}d_{i}^{*}\varepsilon_{i}$ into the sum of two uncorrelated terms,

and observe that the first term on the right-hand side here forms a martingale difference sequence while the second term is asymptotically normal by assumption. Combining these facts, we are able to obtain asymptotic normality of the sum; see the detailed proof in the Appendix.

Like Theorem 1, this theorem establishes two key facts as well. To see both of them, observe that the term $\sigma_{d\varepsilon}^{2}$ appearing in the convergence result (14) can be more explicitly rewritten as

This expression in turn implies that the asymptotic variance of the OLS estimator now depends on the correlation across the vectors $e_{i}$, which means that heteroscedasticity of the regression errors forces OLS variance estimators to be adjusted for correlation across the errors. On the other hand, the expression (15) also demonstrates that it suffices to adjust the variance estimators only for correlation across the random variables $\mu_{A}^{\top}e_{i}$, instead of correlation across the full vectors $e_{i}$. The latter point might seem like a minor technicality but it in fact plays an interesting role in models with heterogeneous treatment effects. Indeed, when $d_{i}\in\{0,1\}$ represents the treatment assignment status, $(y_{i}(1),y_{i}(0))$ represents the pair of potential outcomes with and without treatment, so that $y_{i}=y_{i}(0)+d_{i}(y_{i}(1)-y_{i}(0))$, and $w_{i}$ consists of a non-zero constant only, it follows that $\varepsilon_{i}$ takes the form (7), which matches the setting here with $e_{i}=(y_{i}(0)-\mathbb{E}[y_{i}(0)],y_{i}(1)-y_{i}(0)-\mathbb{E}[y_{i}(1)-y_{i}(0)])^{\top}$ and $\sigma(d_{i})=(1,d_{i})^{\top}$. Hence, $\mu_{A}=(0,\sigma_{d}^{2})^{\top}$, and so the expression for the asymptotic variance of $\sqrt{n}(\hat{\beta}-\beta)$ reduces to the probability limit of

as previewed in the previous section; see expression (8) there. In turn, this expression means that it suffices to adjust OLS variance estimation for correlation across treatment effect, and there is no need to worry about correlation of potential outcomes within any given treatment arm. For example, whenever treatment effects $y_{i}(1)-y_{i}(0)$ are independent across $i$, it suffices to use the usual heteroscedasticity-robust variance formulas, even if regression errors $\varepsilon_{i}$ are correlated. Note, however, that it is necessary to use heteroscedasticity-robust variance formulas even if the treatment effects are i.i.d. (and not just independent), as the formula $s^{2}\left(\mathbf{X}^{\top}\mathbf{X}\right)^{-1}$ is not valid in this case. Moreover, the same results apply even if $w_{i}$’s include non-constant controls as well, as the second component of $e_{i}$’s remains the same in this case.

Finally, we note that as long as the form of correlation across random variables $\mu_{A}^{\top}e_{i}$ is known, estimation of the asymptotic variance based on Theorem 2 is conceptually straightforward. For example, if treatment effects are clustered, it suffices to use Moulton (1986)’s or Liang and Zeger (1986)’s formulas assuming that regression errors are clustered at the same level (even though they could be clustered at a different level because of the clustering of potential outcomes without treatment, for example). For brevity of the paper, we do not provide formal statement of such results, as they are case-specific and depend on the form of the correlation structure, e.g. time series versus cluster versus spatial dependence.