A Risk Aware Two-Stage Market Mechanism for Electricity with Renewable Generation

Electricity markets covering the majority of the US, and most of the industrialized world are operated as multi-settlement markets. These markets are organized in the sense that demand for and supply of energy and ancillary services are matched via a centralized auction mechanism, as opposed to bilateral negotiations over individual transactions [6], [9]. An independent system operator (ISO) runs a given market as a series of multi stage forward markets, and a real time or spot market. Depending upon the region, forward markets are settled days or hours ahead of the intended time of delivery, allowing for provision of cheaper, but relatively inflexible generation. Spot markets open and are settled minutes before delivery in order to balance supply and demand in real time.

While the clearing mechanisms in electricity markets are designed with the objective of maximizing welfare of both producers and consumers, the imperative to increase penetration of renewables and reduce reliance on fossil fuels now strains the existing multi-settlement paradigm. In the past, the primary source of uncertainty in market clearing came from demand side deviations, and such errors reduced to under 5% by the opening of spot markets [7]. Current levels of renewable adoption have already exacerbated typical levels of uncertainty for net demand (demand minus renewable generation), and under official mandates to increase the proportion of energy sourced from renewables this trend will only continue. For example, in California renewables constitute nearly 34% of total retail sales, while a recently passed state bill legislates that 100% of power come from renewables by 2045 [2], [5]. Reliance on renewables to this degree introduces an order of magnitude greater uncertainty in net demand, and necessitates novel market designs to address this challenge.

As a starting point, given that economic dispatch is a multi-settlement process, it makes sense to couple markets across forward and real time stages, and then allow for recourse decisions, rather than settle each stage independently. If a probability distribution for the stochastic generation is known, then maximization of expected welfare is a reasonable objective. This type of problem can be formulated as a two-stage stochastic program, and in fact it is possible to show that stochastic clearing is more efficient than two-settlement systems [6], [8].

There are a couple of issues with the use of expected social welfare as an objective function. In purely mathematical terms, a given realization of a random variable can be quite different from its expectation. Thus optimization of an expectation guarantees little in terms of variation over possible outcomes. Further, real-world observations indicate that economic decision makers are risk averse, or at least act so [1]. Therefore, given increasing levels of generation variability, it is of both theoretical and practical interest to incorporate some notion of risk into market objective functions.

In this paper we study how the introduction of risk preferences into the central objective function affects market operation. We consider a setting with an ISO and multiple generators. The ISO owns a nondispatchable, renewable resource, and the market clears in two stages: a forward stage in which only a forecast for renewable generation is available, and a real time stage, wherein the exact realized renewable generation is known. The generators each own primary and ancillary plants, which may be dispatched in the forward and real time stages, respectively. In the forward market, the ISO schedules primary energy procurements from the generators, and in the real time market purchases ancillary service where necessary. All participants are assumed to be non-strategic price takers. However, while we assume that the generators seek to maximize their expected profit, the ISO is risk averse and minimizes a weighted sum of the expectation and conditional value at risk (CVaR) of its costs. CVaR has over the past two decades become the most widely used risk measure, due to the fact that it is a coherent risk measure, and can be calculated via a convex program [4].

Our main result is the proof of existence of a sequential competitive equilibrium (SCEq) in this risk aware, two-stage market with recourse. In particular, we demonstrate the existence of first and second stage prices such that, given these prices, the generation decisions of the generators in both decisions achieve market clearance in stage two, thus balancing supply and demand. We then specify a two-stage market mechanism which implements the SCEq.

Related work. Numerous past works have studied market and mechanism design and equilibrium outcomes in the two-stage expected welfare maximization, or risk neutral setting, e.g., [3], [15] and [16].

Turning to literature which incorporates risk preferences, several works consider settings in which agents may enter into contracts in order to hedge against risky outcomes. In [11] it is shown that a complete market, wherein all uncertainties can be addressed via a balanced set of contracts, involving agents equipped with coherent risk measures, is equivalent to one in which said agents are risk neutral, and take actions based on a probability density function determined by a system risk agent. The work then investigates necessary and sufficient conditions for existence of an equilibrium consisting of allocations, prices and contracts. Assuming a similar setting in the context of hydro thermal markets, [10] shows that given a sufficiently rich set of securities are available to risk averse agents, that a multi-stage competitive equilibrium may be derived from the solution to a risk-averse social planning solution. [6] investigates difficulties that may arise when risk averse agents maximize their welfare in a market are not complete, including existence of multiple, potentially stable equilibria. Our setting differs from these works in that we have one risk aware customer for multiple risk neutral producers, and we do not allow for transactions between agents outside of the quantities of energy purchased and consumed.

We consider a setting with $N$ conventional generators, and a single renewable generator. An additional entity, the independent system operator (ISO) operates the power grid and plays the role of the social planner (from this point we use the terms interchangeably). For simplicity we consider a single bus network.

We consider a two-stage setting, where generation is dispatched in the first stage (also referred to as day-ahead or DA) and then adjusted in the second stage (real time or RT) to match demand.

Let $D\geq 0$ denote the aggregate demand. This demand is assumed inelastic, i.e., it is not affected by changes in first or second stage prices.

The renewable generator’s output is modeled as a nonnegative random variable $W$, upper bounded by $\overline{W}>0$. We make the following additional assumption on the distribution of $W$.

The probability distribution of $W$ is assumed to be known to all market participants. The marginal cost of renewable generation is zero. The quantity of renewable generation scheduled is denoted $y$.

Conventional generator $i$ has access to a primary plant and an ancillary plant. Generator $i$ schedules its primary plant to produce quantity $x^{G}_{i}\in\mathbb{R}_{+}$ prior to realization of $W$ at cost $a_{i}(x^{G}_{i})^{2}$ where $a_{i}>0$. We assume the primary plant is inflexible, so that its generation level must remained fixed once it is scheduled. After realization of $W$, generator $i$ can activate its ancillary plant to produce level $z^{G}_{i}\in\mathbb{R}_{+}$ at cost $\tilde{a}_{i}(z^{G}_{i})^{2}$ where $\tilde{a}_{i}>0$. Any ancillary generation produced in excess of aggregate demand $D$ can be disposed of or sold in a separate spot market, which we do not consider. We assume that $a_{i}<\tilde{a}_{i}$ for all $i$, $a_{i}\neq a_{j}$ for $i\neq j$, and that $\max_{i}a_{i}<\min_{i}\tilde{a}_{i}$ and $\tilde{a}_{i}\neq\tilde{a}_{j}$ for $i\neq j$.

The generator is compensated for its first stage production $x^{G}_{i}$ at price $P_{1}$. In the second stage, given $W=w$, the generator is compensated for second stage generation $z^{G}_{i}(w)$ at price $P_{2}(w)$.

In a single stage market for a single good, a competitive equilibrium is specified by a price $P$ and quantity $x$ such that, given $P$, producers find it optimal to produce, and consumers find it optimal to purchase, quantity $x$ of the good. Thus, the market clears, i.e., demand equals supply.

To understand the outcome of the two-stage market, we consider a sequential version of competitive equilibrium.

Note that in the SCEq definition, $\overline{z}_{i}^{*}(\cdot)$ and $P^{*}_{2}(\cdot)$ are functions. We now investigate the existence of an SCEq in our two stage, risk aware setting.

Let $\hat{\mu}(w)$ be the Lagrange multiplier corresponding to constraint (10). The Lagrangian for (SPP2) is

$$\begin{split}\mathcal{L}&=\sum_{i}\tilde{a}_{i}\hat{z}^{2}_{i}(w)+\hat{\mu}(w)\left(\hat{y}-w-\sum_{i}\hat{z}_{i}(w)\right),\end{split}$$

(18)

giving, in addition to feasibility, the following optimality conditions for problem (SPP2):

	$\displaystyle 2\tilde{a}_{i}\hat{z}^{*}_{i}(w)-\hat{\mu}^{*}(w)$	$\displaystyle\geq 0\quad\forall\.{i}$		(19)
	$\displaystyle\hat{z}_{i}^{*}\left(2\tilde{a}_{i}\hat{z}^{*}_{i}(w)-\hat{\mu}^{*}(w)\right)$	$\displaystyle=0\quad\forall\.{i}$		(20)
	$\displaystyle\hat{\mu}^{*}(w)\left(\hat{y}^{*}-w-\sum_{i}\hat{z}^{*}_{i}(w)\right)$	$\displaystyle=0\quad\forall\,w,$		(21)
	$\displaystyle\hat{\mu}^{*}(w)$	$\displaystyle\geq 0\quad\forall\,w.$		(22)

Assuming $\hat{y}>w$, $\hat{z}^{*}_{i}(w)>0$ for all $i$, and in particular

$$\hat{z}^{*}_{i}(w)=\frac{\hat{\mu}^{*}(w)}{2\tilde{a}_{i}}.$$

(23)

If $\hat{y}\leq w$ then $\hat{z}^{*}_{i}(w)=0$ for all $i$. Summing (23) over $i$, applying constraint (15), and rearranging gives

$$\hat{\mu}^{*}(w)=2\tilde{a}\cdot\left[\hat{y}-w\right]_{+},$$

(24)

where the constant $\tilde{a}$ is defined as $\tilde{a}:=\left(\sum_{i}\frac{1}{\tilde{a}_{i}}\right)^{-1}$. Therefore

$$\hat{z}^{*}_{i}(w)=\tilde{a}\cdot\frac{[\hat{y}-w]_{+}}{\tilde{a}_{i}}.$$

Summing over $i$ gives the optimal second stage objective value (i.e., the minimum recourse cost given $\hat{x}$)

$$c^{\text{SPP}}_{2}(\hat{x},w)=\sum_{i\in\mathcal{I}}\tilde{a}_{i}\left(\tilde{a}\cdot\frac{[\hat{y}-w]_{+}}{\tilde{a}_{i}}\right)^{2}=\tilde{a}\cdot\left[\hat{y}-w\right]_{+}^{2}.$$

(25)

Therefore, $\text{VaR}_{\alpha}(c^{\text{SPP}}_{2}(\hat{x},W))$ may be expressed as

	$\displaystyle\text{VaR}_{\alpha}\left(\tilde{a}\cdot[\hat{y}-W]_{+}^{2}\right)$
	$\displaystyle=\inf\left\{t\,:\,P\left(\tilde{a}\cdot\left[\hat{y}-W\right]_{+}^{2}\leq t\right)\geq\alpha\right\}$		(26)
	$\displaystyle=\inf\left\{t\geq 0\,:\,P\left(W<\hat{y}-\sqrt{\frac{t}{\tilde{a}}}\right)\leq 1-\alpha\right\}$
	$\displaystyle=\begin{cases}0&\text{if }\hat{y}<F^{-1}_{W}(1-\alpha)\\ \tilde{a}\cdot(\hat{y}-F^{-1}_{W}(1-\alpha))^{2}&\text{if }\hat{y}\geq F^{-1}_{W}(1-\alpha).\end{cases}$		(27)

Given this expression for $\text{VaR}_{\alpha}$, the following lemma gives an explicit expression of $\text{CVaR}_{\alpha}$ for our quadratic cost function setting.

While $\text{CVaR}_{\alpha}(c^{\text{SPP}}_{2}(\hat{x},W))$ is convex in the first stage decision $\hat{x}$ due to (25) and Theorem 1, the upper limit $\hat{\theta}$ of the integral in (LABEL:CVaRintegral) is not a differentiable function of $\hat{y}$, so that the Leibniz integral rule does not directly apply. The next lemma addresses this issue.

Let $\hat{\theta}=\hat{\theta}(\hat{y})=\min\{F^{-1}_{W}(1-\alpha),\hat{y}\}$. Then, problem (SPP) may be written as

	$\displaystyle\min_{\hat{x},\hat{y},\hat{z}(\cdot)\geq 0}$	$\displaystyle\quad\sum_{i}a_{i}\hat{x}^{2}_{i}+(1-\epsilon)\int_{0}^{\hat{y}}\sum_{i}\tilde{a}_{i}\hat{z}^{2}_{i}(w)\,f_{W}(w)\,dw$		(32)
		$\displaystyle\hskip 36.135pt+\frac{\epsilon}{1-\alpha}\int_{0}^{\hat{\theta}}\sum_{i}\tilde{a}_{i}\hat{z}^{2}_{i}(w)\,f_{W}(w)\,dw$
	s.t.	$\displaystyle\quad\sum_{i}\hat{x}_{i}+\hat{y}=D$		(33)
		$\displaystyle\quad\sum_{i}\hat{z}_{i}(w)\geq\hat{y}-w\quad\forall\,w.$		(34)

Locational marginal pricing (LMP) is a commonly used settlement scheme for economic dispatch problems, and previous work has examined extensions of LMPs to problems including two stage markets with recourse. In such models, the LMPs arise as the dual variables to power balance constraints for each stage (in our setting (33) and (34) in (SPP)). Previous work ([3],[15]) has demonstrated that such LMPs support a competitive equilibrium when the ISO or social planner is risk neutral, i.e. when $\epsilon=0$. We state this formally in terms of our setting in the following theorem.

Let $\hat{\lambda}^{*}$ and $\hat{\mu}^{*}(w)$ denote the optimal Lagrange multipliers for constraints (33) and (34), given $W=w$, respectively.

Assuming $\hat{z}^{*}_{i}(w)>0$ for any $i$ (and therefore for all $i$), the second stage price given in (36) can be rewritten in terms of the social planner’s primal decision variables and the level of renewable generation. Rearranging the term in parenthesis in (43) gives

$$\hat{z}^{*}_{i}(w)=\frac{\hat{\mu}^{*}(w)}{\hat{c}_{\epsilon,\alpha}(w)}\quad\forall\,w\leq\hat{y}^{*}.$$

(54)

Summing both sides of (54) over $i$ and using constraint (15) gives

$$\hat{y}^{*}-w=\frac{\hat{\mu}^{*}(w)}{2\tilde{a}\cdot\hat{c}_{\epsilon,\alpha}(w)}\implies\frac{\hat{\mu}^{*}(w)}{\hat{c}_{\epsilon,\alpha}(w)}=2\tilde{a}(\hat{y}^{*}-w).$$

(55)

Thus when $0\leq\epsilon<1$, we have

$$P^{*}_{2}(W)=2\tilde{a}\cdot\left[\hat{y}^{*}-W\right]_{+}.$$

Given that $\hat{x}^{*}_{i}>0$ for any $i$ (and therefore for all $i$), a similar calculation gives

$$\hat{\lambda}^{*}=P^{*}_{1}=2a(D-\hat{y}^{*}),$$

where $a=\left(\sum_{i}\frac{1}{a_{i}}\right)^{-1}$.

We now address the case where $\epsilon=1$, as prices given in the statement of Theorem 6 cannot be applied directly in the case where $\hat{\theta}^{*}<\hat{y}^{*}$. Consider a sequence $\{\epsilon(k)\}$, where $\lim_{k\to\infty}\epsilon(k)=1$. Then, suppressing the dependence of $\hat{\mu}(w)$ on $\epsilon$, and taking the limit as $k\to\infty$ on both sides of (55) gives

$$\begin{split}\lim_{k\to\infty}\frac{\hat{\mu}^{*}(w)}{\hat{c}^{*}_{\epsilon(k),\alpha}(w)}&=2\tilde{a}\cdot\left(\lim_{k\to\infty}\hat{y}^{*}(\epsilon(k))-w\right).\end{split}$$

(56)

The limit $\lim_{k\to\infty}\hat{y}^{*}(\epsilon(k))$ exists, as (SPP) may be solved for the case where $\epsilon=1$, and the optimal solution is unique given our assumptions on the generator cost function form.