Individualized treatment rules
under stochastic treatment cost constraints

The effect of a treatment often varies across subgroups of the population [38, 52]. When such differences are clinically meaningful, it may be beneficial to assign treatments strategically depending on subgroup membership. Such treatment assignment mechanisms are called individualized treatment rules (ITRs). A treatment rule is commonly evaluated on the basis of the mean counterfactual outcome value it generates — what is often referred to as the treatment rule’s value — and an ITR with an optimal value is called an optimal ITR. There is an extensive literature on estimation of optimal ITRs and their corresponding values using data from randomized trials or observational studies [6, 21, 25, 37, 54].

Most existing approaches for estimating ITRs do not incorporate real-world resource constraints. Without such constraints, an optimal ITR would assign the treatment to members of a subgroup provided there is any benefit for such individuals, even when this benefit is minute. In contrast, under treatment resource limits, it may be more advantageous to reserve treatment for subgroups with the greatest benefit from treatment. This issue has received attention in recent work. Luedtke and van der Laan developed methods for estimation and evaluation of optimal ITRs with a constraint on the proportion receiving treatment [20]. Qiu et al. instead considered related problems in settings in which instrumental variables (IVs) are available [34]. In one of the settings they considered, the same resource constraint is imposed as in Luedtke and van der Laan [20] but a binary IV is used to identify optimal ITRs even in settings in which there may be unmeasured confounders. In another setting considered in Qiu et al. [34], the authors considered interventions on a causal IV or encouragement status, and developed methods to estimate individualized encouragement (rather than treatment) rules with a constraint on the proportion receiving both encouragement and treatment [33]. They also developed nonparametrically efficient estimators of the average causal effect of optimal rules relative to a prespecified reference rule. Sun et al. considered a setting in which the cost of treatment is random and dependent upon baseline covariates. They developed methods to estimate optimal ITRs under a constraint on the expected additional treatment cost as compared to control, though inference on the impact of implementing the optimal ITR in the population was not studied [41]. Sun [42] considered a related problem involving the development of optimal ITRs under resource constraints, and established the asymptotic properties of the estimated optimal ITR. Their method appears viable when the class of ITRs is restricted by the user a priori.

In this paper, we study estimation and inference for an optimal rule under two different cost constraints. The first is the same as appearing in Sun et al. [41]. In contrast to earlier work on this setting, we do not constrain the class of ITRs considered and provide a means to obtain inference about the optimal ITR. The second constraint we consider places a cap on the total cost under the rule rather than on the incremental cost relative to control. To our knowledge, the latter problem has not previously been considered in the literature. Both of these estimation problems mirror the intervention-on-encouragement setting considered in Qiu et al. [34] but involve different constraints and a more general cost function.

Similarly as in Qiu et al. [34], the estimators that we develop are asymptotically efficient within a nonparametric model and enable the construction of asymptotically valid confidence intervals for the impact of implementing the optimal rule. We develop our estimators using similar tools — such as semiparametric efficiency theory [31, 51] and targeted minimum loss-based estimation (TMLE) [45, 49] — as were used to tackle the related problem studied in Qiu et al. [34]. Consequently, our proposed estimators are similar to that in Qiu et al. [34]. Therefore, we will streamline the presentation by highlighting the key similarities and focusing on the differences between these related problems and estimation schemes.

The rest of this paper is organized as follows. In Section 2, we describe the problem setup, introduce notation, and present the causal estimands along with basic causal conditions. In Section 3, we present additional causal conditions and the corresponding nonparametric identification results. In Section 4, we present our proposed estimators and their theoretical properties. In Section 5, we present a simulation illustrating the performance of our proposed estimators. We make concluding remarks in Section 6. Proofs, technical conditions, and additional simulation results can be found in the Supplementary Material.

To facilitate comparisons with Qiu et al. [34], we adopt similar notation as in that work. Suppose that we observe independent and identically distributed data units $O_{1},O_{2},\ldots,O_{n}\sim P_{0}$, where $P_{0}$ is an unknown sampling distribution. A prototypical data unit $O$ consists of the quadruplet $(W,T,C,Y)$, where $W\in\mathscr{W}\subseteq\mathbb{R}^{p}$ is the vector of baseline covariates, $T\in\{0,1\}$ is the treatment status, $C\in[0,\infty)$ is the random treatment cost, and $Y\in$ is the outcome of interest. As a convention, we assume that larger values of $Y$ are preferable. We use $V=V(W)\in\mathcal{V}$ to denote a fixed transformation of $W$ upon which we allow treatment decisions to depend. For example, $V$ may be a subset of covariates in $W$ or a summary of $W$ (e.g., BMI as a summary of height and weight). In practice, $V$ may be chosen based on prior knowledge on potential modifiers of the treatment effect as well as the cost of measuring various covariates. We distinguish between $V(W)$ and $W$ because of their different roles. On the one hand, we will assume that the full covariate $W$ contains all confounders and thus is used to identify causal effects, while $V(W)$ might not be sufficient for this purpose. On the other hand, some covariates in $W$ may be expensive or difficult to measure in future applications, and thus implementing an optimal ITR based on a subset $V(W)$ of covariate $W$ may be desirable. In the rest of this paper, we will use the shorthand notation $V$, $V_{i}$ and $v$ to refer to $V(W)$, $V(W_{i})$ and $V(w)$, respectively. We define an individualized (stochastic) treatment rule (ITR) to be a function ${\rho}:\mathcal{V}\rightarrow[0,1]$ that prescribes treatment with probability ${\rho}(v)$ according to an exogenous source of randomness for an individual with covariate value $v$. Any stochastic ITR that only takes values in $\{0,1\}$ is referred to as a deterministic ITR.

In this work, we adopt the potential outcomes framework [27, 39]. For each individual, we use $C(t)$ and $Y(t)$ to denote the potential treatment cost and potential outcome, respectively, corresponding to scenarios in which the individual has treatment status $t$. We use $\operatorname{\mathbb{E}}$ to denote an expectation over the counterfactual observations and the exogenous random mechanism defining a rule, and $\operatorname{E}_{0}$ to denote an expectation over observables alone under sampling from $P_{0}$. We make the usual Stable Unit Treatment Value Assumption (SUTVA) assumption.

We define $C({\rho})$ and $Y({\rho})$ to be the counterfactual treatment cost and outcome, respectively, for an ITR ${\rho}$ under an exogenous random mechanism. We note that if ${\rho}(v)\in(0,1)$ for an individual with covariate $v$, an exogenous random mechanism is used to randomly assign treatment with probability ${\rho}(v)$ and thus $C({\rho})$ and $Y({\rho})$ are random for this given individual. If ${\rho}$ were implemented in the population, then the population mean outcome would be $\operatorname{\mathbb{E}}\left[Y({\rho})\right]$, where we use $\operatorname{\mathbb{E}}$ to denote expectation under the true data-generating mechanism involving potential outcomes $(C(t),Y(t))$ and exogenous randomness in $\rho$. We consider a generic treatment resource constraint requiring that a convex combination of the population average treatment cost and the population average additional treatment cost compared to control be no greater than a specified constant $\kappa\in(0,\infty]$. Consequently, an optimal ITR ${\rho}_{0}$ under this constraint is a solution in ${\rho}:\mathcal{V}\rightarrow[0,1]$ to

maximize

$\displaystyle\operatorname{\mathbb{E}}[Y({\rho})]\hskip 15.00002pt\text{subject to}\hskip 15.00002pt\alpha\operatorname{\mathbb{E}}[C({\rho})]+(1-\alpha)\operatorname{\mathbb{E}}[C({\rho})-C(0)]\leq\kappa\ .$

(1)

Here, $\alpha\in[0,1]$ is also a constant specified by the investigator. Natural choices of $\alpha$ are $\alpha=0$, corresponding to a constraint on the population average additional treatment cost compared to control, and $\alpha=1$, corresponding to a constraint on the population average treatment cost. The first choice may be preferred when the control treatment corresponds to the current standard of care and a limited budget is available to fund the novel treatment to some patients. The second choice may be more relevant when both treatment and control incur treatment costs.

To evaluate an optimal ITR ${\rho}_{0}$, we follow Qiu et al. [34] in considering three types of reference ITRs and develop methods for statistical inference on the difference in the mean counterfactual outcome between ${\rho}_{0}$ and a reference ITR ${\rho}^{\mathcal{R}}_{0}:\mathcal{V}\rightarrow[0,1]$. The first type of reference ITR considered, denoted by ${\rho}^{\mathrm{FR}}$ (${\mathrm{FR}}$=fixed rule), is any fixed ITR that may be specified by the investigator before the study. When $\alpha=0$, it is usually most reasonable to consider the rule that always assigns control, namely $v\mapsto 0$, because the constraint in (1) may arise due to limited funding for implementing treatment whereas the standard of care rule is to always assign control. The second type, denoted by ${\rho}^{\mathrm{RD}}_{0}$ (${\mathrm{RD}}$=random), prescribes treatment completely at random to individuals regardless of their baseline covariates. The probability of prescribing treatment is chosen such that the treatment resource is saturated (i.e., all available resources are used) or all individuals receive treatment, if such a probability exists. Symbolically, this ITR is given by ${\rho}^{\mathrm{RD}}_{0}:v\mapsto\min\left\{1,(\kappa-\alpha\operatorname{\mathbb{E}}\left[C(0)\right])/\operatorname{\mathbb{E}}\left[C(1)-C(0)\right]\right\}$ under the condition that $\operatorname{\mathbb{E}}\left[C(0)\right]\leq\kappa$ and $\operatorname{\mathbb{E}}[C(1)-C(0)]>0$. Although ${\rho}^{\mathrm{RD}}_{0}$ has the same interpretation as the corresponding encouragement rule in Qiu et al. [34], its mathematical expression is different due to the different resource constraint. This rule may be of interest if it is known a priori that treatment is harmless. The third type, denoted by ${\rho}^{\mathrm{TP}}_{0}$ (${\mathrm{TP}}$=true propensity), prescribes treatment according to the true propensity of the treatment implied by the study sampling mechanism $P_{0}$, so that ${\rho}^{\mathrm{TP}}_{0}$ equals $w\mapsto P_{0}\left(T=1\mid W=w\right)$. This ITR may be of interest in two settings. In one setting, ${\rho}^{\mathrm{TP}}_{0}$ satisfies the treatment resource constraint. The investigator may wish to determine the extent to which the implementation of an optimal ITR would improve upon the standard of care. In the other setting, the treatment resource constraint is newly introduced and the standard of care ITR may lead to overuse of treatment resources. The investigator may then be interested in whether the implementation of an optimal constrained ITR would result, despite the new resource constraint, in a noninferior mean outcome.

In this section, we present nonparametric identification results. Though these results are similar to those for individualized encouragement rules in Qiu et al. [34], there are two key differences. First, the form of some of the conditions in Qiu et al. [34] need to be modified to account for the novel resource constraint considered here. Second, two additional conditions are needed to overcome challenges that arise due to this new constraint.

We first introduce notation that will be useful when presenting our identification results and our proposed estimators. For any observed-data distribution $P$, we define pointwise the conditional mean functions $\mu^{C}_{P}(t,w):=\operatorname{E}_{P}(C\mid T=t,W=w)$ and $\mu^{Y}_{P}(t,w):=\operatorname{E}_{P}(Y\mid T=t,W=w)$, where we use $\operatorname{E}_{P}$ to denote an expectation over observables alone under sampling from $P$, and their corresponding contrasts due to different treatment status, $\Delta_{P}^{C}(w):=\mu^{C}_{P}(1,w)-\mu^{C}_{P}(0,w)$ and $\Delta_{P}^{Y}(w):=\mu^{Y}_{P}(1,w)-\mu^{Y}_{P}(0,w)$. We also define the average of these contrasts conditional on $V$ as $\delta^{C}_{P}(v):=\operatorname{E}_{P}[\Delta^{C}_{P}(W)\mid V=v]$ and $\delta^{Y}_{P}(v):=\operatorname{E}_{P}[\Delta^{Y}_{P}(W)\mid V=v]$, and the propensity to receive treatment $\mu^{T}_{P}(w):=P\left(T=1\mid W=w\right)$. Additionally, we define $\nu_{P}(t,v):=\operatorname{E}_{P}\left[\operatorname{E}_{P}\left(C\mid T=t,W\right)\mid V=v\right]$, $\phi_{P}:=\operatorname{E}_{P}[\mu^{C}_{P}(0,W)]$. These quantities play an important role in tackling the problem at hand. Throughout the paper, for ease of notation, if $f_{P}$ is a quantity or operation indexed by distribution $P$, we may denote $f_{P_{0}}$ by $f_{0}$. As an example, we may use $\Delta^{Y}_{0}$ to denote $\Delta^{Y}_{P_{0}}$.

We introduce additional causal conditions we will require, positivity and unconfoundedness. In one form or another, these conditions commonly appear in the causal inference literature [49], including in the IV literature [1, 15, 43, 53].

Equipped with these conditions, we are able to state a theorem on the nonparametric identification of the mean counterfactual outcomes and average treatment effect (ATE) — these results can be viewed as a corollary of the well-known G-formula [36].

In view of Theorem 1, the objective function in (1) can be identified as

$\displaystyle\operatorname{\mathbb{E}}\left[Y({\rho})\right]=\operatorname{E}_{0}\left[{\rho}(V)\mu^{Y}_{0}(1,W)+(1-{\rho}(V))\mu^{Y}_{0}(0,W)\right]=\operatorname{E}_{0}\left[{\rho}(V)\Delta_{0}^{Y}(W)\right]+\operatorname{E}_{0}[\mu^{Y}_{0}(0,W)]\ ,$

and, similarly, the expected cost is identified as $\operatorname{\mathbb{E}}\left[C({\rho})\right]=\operatorname{E}_{0}\left[{\rho}(V)\Delta_{0}^{C}(W)\right]+\operatorname{E}_{0}[\mu^{C}_{0}(0,W)]$. It follows that the optimization problem (1) is equivalent to

maximize

$\displaystyle\operatorname{E}_{0}\left[{\rho}(V)\delta_{0}^{Y}(V)\right]\hskip 15.00002pt\text{subject to}\hskip 15.00002pt\operatorname{E}_{0}\left[{\rho}(V)\delta_{0}^{C}(V)\right]+\alpha\phi_{0}\leq\kappa\ .$

(2)

This differs from Equation 3 defining optimal individualized encouragement rules in Qiu et al. [34]. We now present two additional conditions so that (2) is a fractional knapsack problem [9], thereby allowing us to use existing results from the optimization literature. These conditions are similar to those in Sun et al. [41].

Condition A4 is reasonable if treatment is more expensive than control. When applied to an IV setting as outlined in Remark 2, this condition corresponds to the assumption that the IV is indeed an encouragement to take treatment. This condition is slightly stronger than its counterpart in Sun et al. [41], which only requires that $\Delta^{C}_{0}\geq 0$. This stronger condition is needed to ensure the asymptotic linearity of our proposed estimator in Section 4. Under Condition A4, it is evident that Condition A5 is reasonable because if $\alpha\phi_{0}>\kappa$, then no ITR satisfies the treatment resource constraint in view of the fact that $\operatorname{E}_{0}[{\rho}(V)\delta_{0}^{C}(V)]\geq 0$, whereas if $\alpha\phi_{0}=\kappa$, then only the trivial ITR $v\mapsto 0$ satisfies the constraint and there is no need to estimate an optimal ITR.

Under these two additional conditions, (2) is a fractional knapsack problem [9] in which every subgroup defined by a different value of $V$ corresponds to a different ‘item’. A solution in the special case in which $V(W)=W$ and $\alpha=0$ was given in Theorem 1 of Sun et al. [41]. We now state a more general result with the following differences: (i) the treatment decision may be based on a summary $V$ rather than the entire covariate vector $W$, and (ii) $\alpha$ may take any value in $[0,1]$ rather than only zero. We also explicitly state the randomization probability at the boundary for completeness and clarity. Despite these differences, the result we obtain is similar to Theorem 1 in Sun et al. [41]. Define pointwise $\xi_{0}(v):=\delta^{Y}_{0}(v)/\delta^{C}_{0}(v)$, and write $\eta_{0}:=\inf\{\eta:\operatorname{E}_{0}[I(\xi_{0}(V)>\eta)\delta^{C}_{0}(V)]\leq\kappa-\alpha\phi_{0}\}$ and $\tau_{0}:=\max\{\eta_{0},0\}$.

We also note that the reference ITRs introduced in Section 2 are also identified under the above conditions. In particular, it can be shown that ${\rho}^{\mathrm{RD}}_{0}(v):=\min\{1,(\kappa-\alpha\phi_{0})/\operatorname{E}_{0}[\Delta^{C}_{0}(W)]\}$ and ${\rho}^{\mathrm{TP}}_{0}=\mu^{T}_{0}$.

In this section, we present an estimator of an optimal ITR ${\rho}_{0}$ and an inferential procedure for its ATE relative to a reference ITR ${\rho}^{\mathcal{R}}_{0}$, where $\mathcal{R}$ is any of ${\mathrm{FR}}$, ${\mathrm{RD}}$ or ${\mathrm{TP}}$. The proposed procedure is an adaptation of the method first proposed in Qiu et al. [34, 33].

We begin by introducing some notations that are useful for defining the estimands. We define the parameter $\Psi_{{\rho}}(P):=\operatorname{E}_{P}\left[{\rho}(V)\Delta^{Y}_{P}(W)\right]$ or $\Psi_{{\rho}}(P):=\operatorname{E}_{P}\left[{\rho}(W)\Delta^{Y}_{P}(W)\right]$ for each ITR ${\rho}$ and distribution $P\in\mathscr{M}$, depending on whether the domain of ${\rho}$ is $\mathcal{V}$ or $\mathcal{W}$. Here, we consider the model $\mathscr{M}$ to be locally nonparametric at $P_{0}$ [31]. For $P\in\mathscr{M}$, the ATE of an optimal ITR ${\rho}_{P}$ relative to a reference ITR ${\rho}^{\mathcal{R}}_{P}$ equals $\Psi_{\mathcal{R}}(P):=\Psi_{{\rho}_{P}}(P)-\Psi_{{\rho}^{\mathcal{R}}_{P}}(P)$. We are interested in making inference about $\psi_{0}:=\Psi_{\mathcal{R}}(P_{0})$, where we have suppressed dependence on $\mathcal{R}$ from our shorthand notation.

There is an extensive literature on estimating optimal ITRs and evaluating their performance. Among these works, only a few incorporated treatment resource constraints. In this paper, we build upon Sun et al. [41] and study the problem of estimating optimal ITRs under treatment cost constraints when the treatment cost is random. Using similar techniques as used in Qiu et al. [34], we have proposed novel methods to estimate an optimal ITR and infer about the corresponding average treatment effect relative to a prespecified reference ITR, under a locally nonparametric model. Our methods may also be applied to instrumental variable (IV) settings studied in Qiu et al. [34] when the IV is intervened on.