Using Geographic Load Shifting
to Reduce Carbon Emissions

The recent technology revolution has infiltrated most aspects of modern day life, and has lead to an increase in demand for computing resources. Between 2010 and 2018 there was an estimated 550% increase globally in the number of data center workloads and computing instances [1]. This increased computing demand has commanded a shift from smaller data centers to large-scale, highly optimized and efficient facilities. These facilities are referred to as hyper-scalar data centers and are operated by large technology companies such as Amazon, Facebook, Google, Microsoft and Alibaba. These companies operate vast networks of these data centers which are dispersed geographically throughout the world [2, 3]. Data centers currently consume around 1-2% of electricity

globally [1]. This low percentage is largely due to the increased efficiency of hyper-scalar data centers, which have processed the increased computing demand while only increasing electricity use by an estimated 6% [1]. However, as the scope for further efficiency gains is largely exhausted, it is projected that electricity use for computing will increase rapidly in the future.

Hyper-scalar data centers are large loads on electric power networks with unique characteristics. For example, data center loads can defer when computing tasks are processed or process them at different locations. These properties equip data center operators with the unique ability to participate in both geographic and temporal load shifting. Motivated by recent pledges made by technology companies to reduce their carbon emissions [4, 5, 6], we are interested in understanding how these hyper-scale data centers can effectively interact with electricity markets in a way that minimizes the carbon emissions caused by their computing loads. Start-ups such as Lancium consider computing that adapts to the operation of the electric grid [7], an idea that plays a critical role in the vision of zero carbon cloud computing [8, 9].

When shifting load, data centers interact with electricity markets operated by independent system operators (ISOs). Previous research has examined the impact of integrating data centers and demand response [10, 11, 12, 13, 14, 15, 16] or have considered geographical load shifting to reduce electricity costs [17, 18, 19], including load shifting between different electricity markets [20]. Other work has modelled data center flexibility through the use of virtual links in time and space [21]. Much of this body of work considers the shifting of computing load to reduce the carbon emissions of data centers [22, 23, 19] and increase absorption of renewable energy [24, 25]. There are, however, several major challenges to effective collaboration between the ISO and data centers, such as barriers to the exchange of possibly sensitive information. Another challenge is that ISOs and data centers have different objectives, namely, ISOs focus on minimizing electricity cost while data centers would like to minimize carbon emissions. While it is a common assumption that those two objectives are well aligned, our previous research [26] found that if data centers are specifically targeting the objective of reducing carbon emissions (rather than reducing cost) they can achieve better results by doing the load shifting outside of the electricity market rather than collaborating with the ISO. Since ISOs are unlikely to change the market to directly minimize carbon emissions in the near future, we focus on a method to identify geographical load shifts that reduce carbon emissions within the existing electricity market.

To effectively shift load geographically, we require information about the locational variation in the carbon footprint of electricity, similar to the way locational marginal prices describe the locational variations in the cost of electricity. While data on electricity prices is easily available, similar information on the carbon emissions associated with electricity usage is not. As a result, it is less straight forward to develop metrics that provide good guidance for shifting load to reduce carbon emissions. Previous work in this vein has assumed that prices are directly tied to the fraction of non-renewable energy [27], or considered average carbon emissions for electricity in a region and/or renewable energy curtailment [28, 29, 25]. A benefit to these metrics is that several companies provide the necessary information needed to compute them [30, 31, 32]. A downside is that they all fail to consider necessary aspects of electric grid operation, such as the impact of a marginal change in load and transmission capacity limits. Specifically, there is no consideration for which generators in the system will be providing the additional generation or if is there is sufficient transmission capacity available to deliver the generation. To effectively guide load shifting, we need a measure of marginal emissions [26]. The use of marginal emissions to assess energy efficiency or the potential impact of renewable energy has been considered in [33, 34], proposed to guide renewable energy investments [35, 36, 33], or considered the impact of marginal emissions rates at cogeneration facilities [37]. More recently, a measure of locational marginal carbon emissions was proposed for data center geographical load shifting in [38]. The superiority of this metric over others such as average carbon emissions, curtailment or LMPs for data center geographic load shifting was demonstrated in [26].

While the previous load shifting models in [38, 26] presented promising results, it included several unrealistic assumptions such as the electricity market being solved twice in each time step. Here, we present a model that addresses these issues and also leverages regularization to increase the accuracy of the model. Further, due to the fact that the shifting metric derived in [38] is a linear sensitivity, we expect that at each time step this metric is not necessarily finding the best load shift. For this reason, we present a new benchmark model for optimal load shifting. This bilevel program finds the best load shift a data center can perform, given the current electricity market. In summary, the three main contributions of this paper are the following: 1) Realistic model for data center load shifting: We address the unrealistic assumptions in our previous model by implementing load shifting in a cumulative fashion and incorporating a regularization parameter to increase the accuracy of shifting.

The remainder of the paper is organized as follows. In Section II we review the existing data center driven shifting model, while Section III presents the proposed improvements to this model. In Section IV, we present a new benchmark model for optimal load shifting. Section V demonstrates the efficacy of our model in a case study, and Section VI concludes.

Previous work has considered ISO independent load shifting to reduce carbon emissions via a value called the locational marginal carbon emissions, $\lambda_{\text{CO}_{2}}$ [38]. This value is calculated as a linear sensitivity around the optimal solution to the DC OPF. We give a brief outline of this load shifting model here, but suggest [38] as a more detailed reference.

The cost function (1a) minimizes the cost of generation, where $c_{i}$ is the cost of generation at generator $i$. Constraints (1b)-(1d) constrain the nodal power balance, transmission line and generator capacity constraints respectively. Finally, (1e) sets the voltage angle at the reference node to zero.

To calculate the locational marginal carbon emissions, we first consider a basic optimal solution $x^{*}=[\theta^{*},P_{g}^{*}]\in\mathbb{R}^{n}$ to the DC OPF (1). From sensitivity analysis in linear optimization theory [41], a basic optimal solution can be written as $Ax^{*}=b$, where $A\in\mathbb{R}^{n\times n}$ is a full rank matrix consisting of all the active constraints of (1) at the optimal solution $x^{*}$. In the case of the DC OPF, the rows of $A$ consist of the equality constraints (1b) and (1e) as well as a subset of the inequality constraints (1c), (1d) that are satisfied at equality for $x^{*}$.

A small change in load can be represented as a small change in the right hand side $b$, given by $\Delta b=\begin{bmatrix}\Delta P_{d}&0\end{bmatrix}^{T}$. Assuming that the change is small enough as not to alter the set of active constraints, we compute the optimal change in generation as $A\cdot\Delta x=\Delta b$ where $\Delta x=[\Delta\theta\ \Delta P_{g}]$. This gives rise to the linear relationship

If we denote the matrix consisting of the last $|\mathcal{G}|$ rows and first $|\mathcal{N}|$ columns of $A^{-1}$ by $B$, this gives the linear relationship between load and generation changes, $\Delta P_{g}=B\cdot\Delta P_{d}$.

Consider the cost vector $g\in\mathbb{R}^{|\mathcal{G}|}$ that measures the carbon emissions of each generator per MW. Specifically, the $i$th component of $g$, is the carbon intensity of generator $i$. Multiplying each side of $\Delta P_{g}=B\cdot\Delta P_{d}$ on the left by $g$ gives us the change in carbon emissions

where $\lambda_{\text{CO}_{2}}=g\cdot B$. Intuitively, the $k$th component of $\lambda_{\text{CO}_{2}}$ measures how an increase of $1$ MW of load at node $k$ will affect the total carbon emissions of the system.

We let $\mathcal{C}$ denote the set of data center loads that can geographically shift load and consider optimization variables that denote the change in load at data center $i$, $\Delta P_{d,i}$, and the shift in load from data center $i$ to $j$, $s_{ij}$. The geographic load shifting optimization problem is given by:

The objective value (4a) minimizes the change in carbon emissions as a function of the change in load, (4b) enforces that the change in load at a given data center is equal to the total load shifted in minus the total load shifted out, while (4c) says the sum of all load shifts must be zero. Constraint (4d) limits the amount each data center can shift as a fraction $\epsilon$ of the data center capacity Cap. Constraint (4e) puts an upper bound $M_{ij}$ on the total capacity at each data center and (4f) limits how much load data center $i$ can send to data center $j$.

The above model has several drawbacks. First, it is unrealistic to assume that the ISO resolves the market clearing twice, once before and once after the shifting has happened. Second, since the model is linear, we tend to see large load shifts even with small differences in $\lambda_{\text{CO}_{2}}$ between data center locations. Since $\lambda_{\text{CO}_{2}}$ is a local sensitivity factor that is only accurate near the previous optimal solution, these large shifts lead to inaccurate results that sometimes increase carbon emissions. To address these issues, we introduce two improvements to the model: cumulative load shifting and regularization.

Our next contribution is to introduce a new model to benchmark the data center driven shifting model. Since the shifts provided by $\lambda_{\text{CO}_{2}}$ are calculated by a linear sensitivity, they can be inaccurate, even giving shifting profiles that increase carbon emissions.

The problem of identifying the optimal load shift data center operators should employ to minimize carbon emissions can be modelled as a bilevel linear program. The upper level problem identifies the optimal choice of load shift $\Delta P_{d}$ to minimize carbon, i.e.,

Here, the last constraint represents the set of feasible load shifts from the data center perspective, i.e., $\mathcal{P}$ is the polytope of permissible load shifts defined by the constraints in (4).. The first constraint states that the generation values $P_{g}^{*}$ is the solution to the lower level optimization problem (DC-shift). This problem is a version of the standard DC OPF (1) where the nodal balance constraints include the change in demand. Formally we write it as

As in $(\lambda_{\text{CO}_{2}}$-shift), we also consider cumulative load shifting in (Opt-shift). Specifically, at each data center $\ell\in\mathcal{C}$, at time step $t$ we assume the load $P_{d,\ell}$ in (DC-shift) reflects the sum of new load $P_{d,\ell}$ from time $t$ and the load shift $\Delta P_{d,\ell}$ from time $t-1$. Herein out when we refer to the model (Opt-shift) we assume it is employed with this cumulative load shifting.

We next analyze the efficacy in carbon reduction of $(\lambda_{\text{CO}_{2}}$-shift) versus (Opt-shift).

In this paper we presented an improved model for data center load shifting to reduce carbon emissions. This model shifts load independently of ISO collaboration via a measure known as locational marginal carbon emissions. We built on existing work, but made several improvements to increase realism and accuracy of our model. We also proposed a new benchmark model which gives the best load shift at each time step, and compared the results. The main conclusion from the paper is that smaller load shifts, limited by regularization and shifting caps, are quite effective in reducing carbon emissions. Larger load shifts tend to decrease accuracy of the model and produce less carbon savings. Further, while our benchmark model is able to achieve large carbon reductions, it also significantly increases cost.

This paper demonstrates many natural directions for future work. First, this work shows that shifting load in a greedy way, i.e. shifting to reduce the maximal amount of carbon at each time step, is not necessarily the best approach if current load shifts will impact the load profile in future time steps. This demonstrates that forecasting future load and generation patterns is important to obtain better solutions. Finally, the information needed to calculate the locational marginal carbon emissions is currently not made publicly available. Therefore, finding ways to infer and predict $\lambda_{\text{CO}_{2}}$ at the data center nodes from publicly available data is needed in order to implement $(\lambda_{\text{CO}_{2}}$-shift) in practice.