The Causal Effect of the Two-For-One Strategy in the National Basketball Association

This study leverages detailed play-by-play data from the 2018-19 and 2021-22 NBA seasons, including variables such as player lineups, betting odds, and player ratings, to rigorously analyze the effectiveness of the two-for-one strategy. By framing the two-for-one strategy as a causal inference problem, we provide a formal statistical model that allows for reliable estimation of treatment effects while controlling for confounding factors. The methods used in this analysis—inverse probability weighting, matching, causal forests, and RATE—are all designed to ensure that our results are robust and generalizable to other contexts in sports analytics.

The main contributions of this paper are as follows:

1. 1.

   We frame the two-for-one strategy as a causal inference problem, translating a complex sports decision into a statistical model that can be rigorously analyzed.

2. 2.

   We estimate the average treatment effect of the two-for-one strategy, providing quantitative evidence of its effectiveness in NBA games.

3. 3.

   We investigate potential heterogeneity in the strategy’s impact using the causal forest framework, finding that the strategy’s effectiveness appears relatively consistent across various game contexts.

Our findings contribute to sports analytics by offering data-driven insights into an influential NBA strategy, providing coaches and analysts with valuable, evidence-based guidance for optimizing in-game tactics.

The structure of this paper is as follows. Section 2 presents definitions and background on the two-for-one strategy and details the data set used in our analysis. Section 3 outlines the causal methodology employed to assess the impact of this strategy. Section 4 presents the results obtained from analyzing two seasons of NBA data, highlighting key findings on the effectiveness of the two-for-one approach. Section 5 concludes with a brief discussion.

In order to determine the causal effect of the TFO strategy in the NBA, we need a well-defined treatment. That is, we need to be able to classify scenarios during an NBA game as those in which a team enacted the TFO strategy, and those in which they did not. This is not a trivial matter. Teams do not announce that they are attempting a TFO, and it is unclear how attune players are to the game and shot clock during any particular play. Thus we rely on a combination of time and play outcomes in order to identify TFO opportunities (TFO-O). These are scenarios when a team comes into possession of the ball with enough time left in the quarter such that taking a shot or initiating offense relatively early in the shot clock leads to a increased chance to get an extra possession, while waiting until the end of the shot clock to do these things increases the chance the opposing team will have the last possession. We then categorize those TFO-O into attempts (TFO-A) and non-attempts (TFO-NA).

Several important characteristics of NBA game play inform these definitions. The shot clock in the NBA is 24 seconds, meaning that from the time a team gains possession of the ball after a turnover, rebound, or made basket, the team has 24 seconds in which to attempt a shot. If a team gains possession with less than 24 seconds left in the quarter, the shot clock is turned off and the team in possession can attempt to have the last possession of the quarter by waiting until the very end to take a shot. Therefore, if a team wants to improve its chances of gaining the last possession of the quarter, it should hope that the opposing team gains possession with more than 24 seconds left.

However, it can take up to a few seconds for possession to change, particularly after a missed shot. As an example, suppose a player on Team A takes a shot with 6 seconds left in the quarter. It takes one second for the ball to get from the player’s hand to the rim, another second to bounce on the rim a couple of times before coming to the ground, and a third second before a player on Team B possesses it, now with only 3 seconds left in the quarter. Furthermore, if there is too little time left in the quarter when a team gains possession, that team has a very small chance of scoring any points.

With these characteristics in mind, we set a limit of 28 seconds left in the quarter, and say that if a team takes a shot after this point and the possession changes, it is unlikely that they will get a quality possession at the end of the quarter. Working backwards, then, a team needs to gain possession of the ball with enough time before the 28 seconds mark in order to potentially attempt a reasonable shot by this point. We set this lower limit to be 35 seconds. We choose 43 seconds as an upper limit on when a team can gain possession to exclude those scenarios where a team can get to this optimal shot time simply by waiting until the shot clock for their possession is close to 0. Thus we define a TFO-O as one in which a team comes into possession of the ball with between 35 and 43 seconds left in the quarter.

If a team presented with a TFO-O takes a shot or is fouled while taking a shot with at least 28 seconds to go in the quarter, we consider that a clear TFO-A. Other play results are harder to categorize. We also consider an opportunity a TFO-A if the team is fouled with at least 28 seconds to go in the quarter, with the rationale being that fouls more often occur when a team is trying to initiate offense. On the other hand, we consider an opportunity to be a TFO-NA if the team maintains possession until there are less than 28 seconds left in the quarter without taking a shot or being fouled.

We remove all other play outcomes that occur during a TFO-O from consideration rather than defining them as a TFO-A or a TFO-NA. The most common of these is a turnover. Turnovers can happen while a team is initiating offense, such as a charge foul, or just trying to hold the ball until the end of the shot clock, such as a steal that happens far away from the basket. Due to the lack of information on turnovers available in play-by-play data, we choose not to put them into either the TFO-A or TFO-NA categories. The same is true for other, less common plays such as a jump ball, a defensive three-second violation, and a kicked ball violation.

We define our response variable as the Post Opportunity Difference (POD), denoted by $Y_{i}$. The POD quantifies the effectiveness of the TFO strategy by measuring the difference in score differential from the time of the TFO-O to the end of the quarter. To calculate $Y_{i}$, we first determine the score difference between the teams at the time of the TFO-O, denoted $Y_{i,TFO-O}$. Next, we record the score difference at the end of the quarter, denoted $Y_{i,EOQ}$. The POD is then defined as:

A positive value of $Y_{i}$ indicates that the team increased its score margin by the end of the quarter. Conversely, a negative $Y_{i}$ implies that the team’s score margin decreased after the TFO-O. Thus if the TFO strategy is effective the POD would be higher for teams that attempt a TFO than those who do not.

The main source of data we use is the play-by-play data from the NBA. This is a list of all the different plays that took place in an NBA game as well as supplemental information such as the players involved in the play, the score of the game at the time of the play, the quarter the play took place in, and the amount of time left in that quarter. We obtain play-by-play data for each regular season game in the 2018-2019 and 2021-2022 seasons using the nbastatR package (Bresler, 2024). The 2018-2019 season was the last regular season before the Covid-19 pandemic altered the NBA schedule, and the 2021-2022 season was the first full season after the pandemic, making these natural seasons to consider.

Using this play-by-play data, we identify all TFO-O in the first three quarters of each NBA regular season game. We choose to exclude the fourth quarters because the behavior of NBA games is often much different at the end of those quarters. If the game is not close, teams are no longer necessarily worried about maximizing their possessions because of the feeling that nothing they do will affect the outcome. If the game is very close, teams tend to foul more to try to regain possession. We believe that excluding these scenarios allows us to better isolate the treatment. For the 2018-2019 season, there were a total of 1036 TFO-A and 529 TFO-NA. For the 2021-2022 season there were a total of 950 TFO-A and 462 TFO-NA.

To enhance our analysis with relevant covariates, we draw from several supplemental data sources. First, to gauge the relative strength of the teams at the time of each game, we incorporate betting odds, which indicate the favored team and the expected point spread. These odds effectively summarize expert assessments of each team’s strength. Additionally, to measure the projected pace of play, we include betting totals, which provide a prediction of the total points expected to be scored in a game. Both the betting odds and totals are sourced from the Sportsbook Reviews archive (Sportsbook Reviews, 2023).

For player-specific information, we use individual player ratings from the NBA 2K video game series (HoopsHype, 2018, 2021), which provide relative strength assessments for each player. We obtain starting lineups from BasketballReference.com (Basketball Reference, n.d.) and use this information, combined with play-by-play data, to determine which players are on the court at any given time. These covariates allow us to account for variations in team and player strength as well as game tempo, enhancing the robustness of our analysis. Table 1 provides the name and definitions of the variables we use.

In the following section, we describe the causal inference methodology we employed, detailing our computation of the average treatment effect and our investigation into treatment effect heterogeneity.

The consistency assumption implies that there is a well-defined treatment with no hidden variations. In our setting we recognize that not all TFO-A look the same, as highlighted in the previous section. Some end in a shot while others end in a foul. In some cases the team that had the TFO-A gets the ball back, and in other cases it does not. However, because these differences are a part of the definition of our treatment, they are not hidden and do not form an egregious violation of the consistency assumption.

The no interference assumption says that the outcome for a particular unit does not depend on the treatment assignment of any other unit. Mathematically, this means that for any individual $i$,

where $W_{-i}$ represents the treatment vector excluding $i$. This assumption would be violated in our setting if the success of a particular team’s TFO-A was affected by whether another TFO-O in the study was an attempt or a non-attempt. It seems unlikely that this would be true if the two units are from separate teams. It is possible due to the season-long nature of the data that if a particular team has a large number of TFO-A relative to TFO-NA early in the year, the players may practice last-second shots more because they have been experienced them more in the course of play, and thus they could improve the outcome of the TFO-A later in the year. However, players already practice these types of shots because of their importance to game play, and it seems unlikely that the amount of time they spend practicing such shots substantially depends on how often they attempt a TFO strategy. Thus the no interference assumption seems to reasonably hold for our setting.

Conditional exchangeability implies that, given a set of covariates $X$, the treatment assignment $W$ is independent of the potential outcomes

This allows us to assume that treated and untreated individuals with the same values of $X$ have comparable distributions of potential outcomes, thus enabling causal comparisons. Suppose our set of covariates consists of the score margin and the time left in the quarter. Then conditional exchangeability would say, for example, that TFO-O scenarios where a team is down by 4 with 40 seconds left in the quarter are the same, on average, for both the TFO-A and TFO-NA groups. The key to reasonably satisfying this assumption is choosing a set of covariates rich enough to include any major potential confounding variables.

The list of covariates we consider is given by Table 1. When considering whether teams convert a TFO-O into a TFO-A, perhaps the most obvious factor is how much time is left in the quarter. We also believe that there may be quarter specific effects and thus include indicators for the first three quarters. The scoring margin at the time of the TFO-O is included as a team may feel more of a need to maximize possessions if it is trailing in the game. The quality of the lineup on the floor for both teams is also important and is characterized by the different rating covariates. We use the betting spread to account for how close a game is expected to be, and the betting total to account for how high or low scoring the game is expected to be. We believe these are the most important sources of potential confounding and thus that conditional exchangeability is reasonably satisfied. It is also common to check this condition by looking at the balance of the covariates in the groups created by the methods discussed below, which we do in Section 4.1.

The final assumption necessary is the positivity assumption, which says that each TFO-O must have a positive probability, conditional on our set of covariates, of being in either the treatment or control group i.e.,

There is a known trade-off between positivity and conditional exchangeability. If you condition on too many covariates, you may satisfy the latter but not the former. If you do not condition on enough covariates, the opposite may be true. We feel that we have found a good balance between the two by carefully selecting a small but rich set of covariates. We perform a visual check of the positivity assumption in Section 4.1.

To estimate treatment effects we make use of two common causal inference methods: inverse probability weighting (IPW) and matching. Both of these methods rely on the concept of a propensity score (Rosenbaum & Rubin, 1983), which is defined as the probability of receiving treatment given baseline covariates. For a binary treatment $W\in\{0,1\}$, the propensity score denoted by $e(X)$ is given by

This scalar summary of multiple covariates facilitates covariate balance between treated and control groups, aiming to replicate the distributional balance of a randomized experiment. Propensity scores are generally unknown quantities but can be estimated using logistic regression or machine learning methods such as random forests or gradient boosting (Hastie et al., 2009), which can handle complex relationships between covariates and treatment assignment (Lee et al., 2010). For our analysis we employ the covariate balancing propensity score method of Imai & Ratkovic (2013), which improves performance by leveraging both the goal of modeling treatment assignment and that of balancing the covariates. In practice, propensity scores are often computed in software such as R, using packages like MatchIt, which provides matching methods, or WeightIt, which computes weights for propensity score weighting (Stuart et al., 2011; Greifer, 2024b). Under the assumptions of conditional exchangeability, positivity and consistency, the propensity score allows for unbiased estimation of the ATE (Imbens & Wooldridge, 2009; Stuart, 2010).

Propensity scores are used to estimate treatment effects in several ways. IPW assigns weights to individuals to create a pseudo-population where the distribution of covariates is balanced between treatment groups. For an individual $i$ with covariates $X_{i}$ when the ATE is the estimand of interest, weights are computed as

where $\hat{e}(X_{i})$ is the estimated propensity score. This weight is larger for individuals in groups with a lower probability of treatment assignment, adjusting for selection bias by up-weighting underrepresented individuals (Hirano et al., 2003). If conditional exchangeability holds in the true population, then marginal exchangeability holds in the pseudo-population. Thus the ATE can then be estimated by the weighted difference in outcomes:

where $Y_{i}$ denotes the observed outcome. If the ATT or ATC are the estimands of interest, the weights are simply modified so that the distribution of the group of interest is unchanged, leading to

This inverse probability weighting (IPW) estimator can be implemented in R using the survey package, which enables the application of IPW techniques to survey data structures, and twang, which supports estimation and diagnostics for IPW models with large datasets (Robins et al., 2000; Cefalu et al., 2024)

Propensity score matching pairs treated and control individuals with similar propensity scores, aiming to create a balanced sample for which treatment and control groups are comparable on observed covariates. When estimating the ATT, each treated unit $i$ is matched to a control unit $j$ with a propensity score as close as possible to $e(X_{i})$. When estimating the ATC, each control unit $i$ is matched to a treated unit $j$ with a propensity score as close as possible to $e(X_{i})$. Matching can be done using nearest-neighbor matching, which minimizes the absolute difference between scores; caliper matching, which restricts matches to pairs within a predefined range; or optimal matching, which minimizes the overall distance across matched pairs (Austin, 2011; Rosenbaum, 2002). If there are many more observations in one of the groups, several units in the larger group may be matched with a single unit in the smaller group. Typically matching is done without replacement, although with replacement matching is also possible.

Again, if conditional exchangeability holds in the true population, then marginal exchangeability holds in the matched population, and the treatment effect can be estimated as the difference in the treatment and control group averages in the matched population. Thus for the ATT we have

where $n_{1}$ is the number of treated individuals and $\mathscr{M}(i)$ represents the matched control units for treated unit $i$. For the ATC we can simply swap the treatment and control group in the above formula. This method can effectively balance covariates if quality matches can be found, making it a robust approach for estimating causal effects when assumptions hold. In R, the MatchIt package provides flexible options for implementing nearest-neighbor, caliper, and optimal matching, along with diagnostic tools for balance checking (Stuart et al., 2011).

It is important to note the methodological distinction between these two methods. In IPW, each observation is reweighted based on the estimated propensity score, allowing us to balance the treated and control groups in expectation. In contrast, propensity score matching directly pairs treated units with control units that have similar propensity scores, creating a matched sample for estimation. Both approaches aim to reduce bias in estimating the treatment effect, but matching emphasizes the creation of comparable subgroups, whereas weighting adjusts the contribution of all observations based on their propensity scores. Both methods are widely used in causal inference applications. See Hernan & Robins (2024) for a theoretical account of both methods, and Shiba & Kawahara (2021) for a recent practical comparison. We present effect estimates from both methods in Section 4.1.

The ATE provides insight into the overall causal effect for a population. However, treatment effects often vary across individuals within the population. The presence of heterogeneity in treatment effects can obscure important subpopulation dynamics if only the ATE is considered. Identifying and analyzing these differences allows for more tailored interventions, improving both individual and aggregate outcomes. Furthermore, examining treatment effect heterogeneity can enhance our understanding of how covariates interact with the treatment, providing deeper insights into causal mechanisms. This motivates the need for methodologies that move beyond the ATE to uncover nuanced variations in causal effects across different subgroups.

The Conditional Average Treatment Effect (CATE) (Athey & Imbens, 2016) is a statistical measure used to capture the heterogeneity in treatment effects across different subgroups within a population. It extends the concept of the ATE, which calculates the expected difference in outcomes due to a treatment across the entire population, by conditioning on specific covariates $X$. This approach is particularly useful in causal inference to identify how treatment efficacy varies across different individuals or groups, allowing for more personalized interventions.

Mathematically, the CATE is defined as:

By conditioning on $X=x$, CATE captures the expected treatment effect for individuals with characteristics $X=x$. Assuming conditional exchangeability and consistency, we can estimate CATE from observational or experimental data by modeling the outcome response surface.

Classical approaches to heterogeneous treatment effect (HTE) estimation often rely on subgroup analysis, where subgroups are defined by observed covariates. However, traditional regression-based methods may be limited by the need to pre-specify interactions or assume homogeneity within subgroups. Recent advances have thus focused on machine learning techniques, which allow for more flexible, data-driven discovery of treatment effect heterogeneity. For instance, Athey & Imbens (2016) introduced causal trees for identifying subpopulations with distinct treatment effects, while Wager & Athey (2018) extended this to causal forests, a method based on random forests that provides both individual-level treatment effect estimates and uncertainty quantification.

Propensity score-based methods are also adapted for HTE analysis, such as subclassification on propensity scores within stratified groups (Rosenbaum & Rubin, 1983). However, propensity-based models assume covariate balance is achieved within each stratum, which may not hold in practice for high-dimensional data. To address this, flexible machine learning models, including generalized additive models and Bayesian Additive Regression Trees (BART), are commonly used to improve balance across strata and yield robust HTE estimates (Hill, 2011).

In our analysis, we use the causal forest framework to compute heterogenous treatment effects.

Once the causal forest model is built, it is essential to test for significant heterogeneity in the estimated causal effects. One approach to evaluate this is through a Calibration Test (Chernozhukov, Demirer, Duflo & Fernandez-Val, 2018), which is conducted as follows:

We fit a linear regression where the target (outcome) variable serves as the dependent variable, and the independent variables are: (1) the overall treatment effect, denoted as mean forest prediction, and (2) the predicted conditional average treatment effect (CATE) based on the out-of-bag trees, denoted as differential forest prediction.

If significant heterogeneity exists in the causal effects, the coefficient associated with differential forest prediction will be statistically different from zero. This indicates that the predicted heterogeneity in treatment effects contributes meaningfully to explaining the outcome variability.

The Rank Weighted Average Treatment Effect (RATE) (Yadlowsky et al., 2024) provides a more formal approach to testing for heterogeneity in causal effects. This method begins with a prioritization rule based on covariates, $X_{i}$. The prioritization rule, $S(X_{i})$, assigns a score to each sample, where a higher score reflects a higher expected benefit from the treatment. In other words, samples predicted to gain more from the treatment receive higher prioritization scores.

The method then measures the incremental benefit of administering the treatment to the top $q$% of samples according to their prioritization scores, a concept captured by the Target Operating Curve (TOC). Let $F$ be the distribution function of $S(X_{i})$ and for $q\in(0,1]$, the TOC is defined as:

At $\text{TOC}(1)$, the difference between the two expectations becomes zero, as the entire sample is included. The $\text{TOC}(q)$ is estimated by comparing the overall average treatment effect (ATE) with the ATE for the subset of samples whose prioritization scores exceed a given threshold, analogous to the receiver operating characteristic (ROC) curve.

RATE is the area under the curve of the TOC curve i.e.,

The RATE value lies between 0 and 1. A good prioritization rule, one that accurately reflects treatment benefit, will result in a RATE value close to 1. Conversely, a poor rule will yield a RATE value near 0.

To test for overall heterogeneity, we can use the CATE estimates as the prioritization rule, assigning higher scores to observations with higher predicted CATE values. If significant heterogeneity is present, the RATE will be close to 1.

Alternatively, we can assess heterogeneity across a specific variable by basing the prioritization rule on that variable rather than the CATE estimates. For instance, we could use $S(X_{i})=X_{i1}$ to rank samples by a particular covariate. In our case, we are interested in evaluating treatment effect heterogeneity when ranking is based on player rating and opponent player rating.

We now apply the methods discussed in the previous section to our data and present the results in the following section.

In our data set there are approximately twice as many treated observations as control observations each year. Therefore, when performing propensity score matching, it is more natural to estimate the average treatment effect on the control group, the ATC. For the sake of consistency, we estimate the ATC when using IPW as well. For teams that do not attempt a TFO in a certain situation, the ATC is a measure of how their average POD would have been different if they had attempted the TFO. This is a natural estimand for the goal of convincing teams that do not currently emphasize the TFO strategy to begin to do so. A positive ATC indicates an increase in the point differential, implying that the TFO strategy is beneficial. Conversely, if the ATC is not significantly different from zero or is negative, it suggests that the TFO strategy is ineffective.

The ATC is computed separately for each season, as well as for the combined dataset that includes both seasons. Estimating the ATC separately provides insight into the stability of the treatment effect across seasons, while combining the data allows for a larger sample size, leading to narrower confidence intervals. We report both the point estimates of ATC and the corresponding 95% confidence intervals.

We perform all calculations in R (R Core Team, 2024). To estimate the ATC using IPW, we employ the R package WeightIt to obtain the weights using covariate balancing propensity scores and the survey package to obtain point estimates and confidence interval. For propensity score matching, we obtain matches using the MatchIt package with 1:1 optimal pair matching without replacement, again based on covariate balancing propensity scores. We use the marginaleffects R package (Arel-Bundock et al., 2024) to obtain point estimates using g-computation as well as interval based on cluster-robust standard errors.

The average treatment effect provides valuable insight, indicating that the TFO strategy is beneficial on average. However, there is a need to explore how this effect varies across different values of the covariates. Understanding this heterogeneity can help identify specific scenarios where the TFO strategy should or should not be attempted, thereby informing team strategy.

To this end, we investigate treatment effect heterogeneity to address the following key questions. First, is there significant heterogeneity in the treatment effect? Second, which variables contribute to this heterogeneity? We employ the causal forest framework to answer these questions.

Before fitting the causal forest model, we augment the dataset with three additional variables: the difference in the maximum ratings between the two teams, the difference in the median ratings between the two teams, and the absolute value of the score margin. Initially, we run the causal forest model with all available variables. We then compute the variable importance measures and retain only those variables that cumulatively account for 95% of the total importance, eliminating three variables in the process. This step helps control the standard error.

As an initial check, we estimate the ATE using the causal forest. The estimated ATE is 0.61, with a standard error of 0.11, relatively close to the results obtained from IPW and matching methods.

This analysis sets the foundation for a deeper exploration of heterogeneity in treatment effects, guiding future strategy development.