Hierarchical Spatio-Temporal Uncertainty Quantification for Distributed Energy Adoption

In recent years, the global energy landscape has experienced a transformative shift, primarily characterized by an extraordinary increase in renewable energy sources [1]. One of its primary contributors is distributed energy resources (DER), a decentralized approach to power generation that includes a variety of technologies such as solar panels, wind turbines, and small-scale hydroelectric systems [2].

While DER deployment has accelerated rapidly, it introduces significant spatio-temporal variability and uncertainties that vary across regions and evolve over time. Understanding these uncertainties is crucial for effective energy management [3], facilitating DER integration into existing power grids [4], identifying grid enhancement opportunities [5], and planning for future energy demands [6]. More importantly, such uncertainty quantification (UQ) must also be accessible at multiple spatial scales to offer a more comprehensive overview of the future and support their downstream strategic decision-making. For example, granular circuit-level predictions enable precise operational decisions, such as real-time load balancing and resource allocation. Meanwhile, aggregated substation-level forecasts provide the broader perspective needed for long-term infrastructure investments, including capacity enhancement and resilience improvements.

However, achieving multilevel uncertainty quantification presents two main challenges: First, jointly predicting multiple spatial units (e.g., circuits) can result in overly conservative uncertainty intervals, potentially diminishing their practical utility. Excessively wide prediction intervals fail to provide actionable insights for specific operational needs. Second, while aggregating these predictions from circuit to substation level may seem straightforward, doing so without careful adjustment can result in prediction intervals that do not fully capture the true underlying uncertainty at the aggregate level. Simple summation of circuit-level uncertainties often overestimates variability, leading to either inflated risk assessments or misleading confidence in grid performance metrics.

To address these challenges, we propose a hierarchical spatio-temporal model based on conformal prediction [7] to predict the spatio-temporal uncertainty in DER growth. Our approach begins by generating circuit-level growth predictions along with corresponding uncertainty measures, and then aggregates these predictions to the substation level based on the grid topology. A key methodological contribution of our method is a novel non-conformity score tailored to this multilevel spatio-temporal predictive task. Our results show that this method maintains both statistical validity and efficiency across multiple spatial levels, ensuring that prediction intervals remain informative and practically useful.

Finally, we apply our method to a real-world dataset containing rooftop solar panel installation records in a major U.S. city over the last decade, collected in collaboration with the local utility. Our findings demonstrate the method’s empirical effectiveness and show remarkable improvements over the state-of-the-art approaches, especially in reducing the width of prediction intervals while maintaining coverage. This highlights the utility of our model in providing stakeholders with actionable insights for future strategic decisions.

Forecasting demand for DER has gained increased attention in recent years, particularly in predicting power and electricity consumption [8]. Driven by the high-stakes nature of their downstream operations, a wide range of UQ techniques have been explored, such as probabilistic forecasting [9], interval forecasting [10], and deep learning approaches [11]. Due to the unique distributed network topology structure of these renewable energy infrastructures, much work has also explored how these inherent hierarchies can be respected or exploited in various forecasting applications including electricity grid management [12] and power generation forecasts [13].

Our work extends further on these works by incorporating conformal prediction (CP) [7], a distribution-free UQ approach. We consider the distribution shift [14] induced by time series by adopting the line of sequential CP framework [15]. Additionally, our method gained inspiration from the Probabilistic Conformal Prediction [16, 17], allowing our method to tailor to more complex DER forecasting problems by integrating probabilistic models. Our work is also related to hierarchical time series prediction [18].

Consider a utility network consisting of $n$ distribution circuits and $m$ distribution substations. Each substation serves as a “hub”, connecting and coordinating multiple circuits within the network. The network topology between circuits and substations is defined by a matrix $\mathbf{C}\coloneqq(c_{ij})\in\{0,1\}^{n\times m}$, where $c_{ij}=1$ indicates circuit $i\in\mathcal{I}$ is associated with substation $j\in\mathcal{J}$, and $c_{ij}=0$ otherwise.

For each circuit $i$ at time $t$, let $Y_{it}\in\mathbb{Z}_{+}$ represent the number of DER installations (e.g., rooftop solar panels). Additionally, let $Z_{it}\in\mathbb{R}^{p}$ denote a set of demographic covariates associated with circuit $i$ at time $t$; these covariates may include factors like population density, average income, or degree of urbanization in the circuit’s vicinity. To simplify notation, define $Y_{t}\coloneqq(Y_{it})_{i=1}^{n}$ as the vector capturing DER installation counts across all circuits at time $t$, and $Z_{t}\coloneqq(Z_{it})_{i=1}^{n}$ as the corresponding vector of demographic covariates. Suppose we have a calibration dataset, denoted by $\mathcal{D}=\{(x_{t},y_{t})\}_{t=1}^{T}$, where $y_{t}$ represents the DER installation counts at time $t$, and $x_{t}\coloneqq(\{y_{\tau}\}_{\tau<t},z_{t-1})$ represents all observed predictor variables, including both the historical DER installation counts across circuits up to time $t-1$ and the demographic covariates $z_{t}$ at time $t-1$.

Our objective is to construct prediction intervals for each circuit at future time $T+1$, defined by lower bounds $\widehat{L}\coloneqq(\widehat{L}_{i})$ and upper bounds $\widehat{U}\coloneqq(\widehat{U}_{i}$). Let $Y_{T+1}$ represent the DER installation counts across all circuits for the next time step, and $\mathbf{C}^{\top}Y_{T+1}$ represent aggregated DER installation counts at the substation level. For a specified confidence level $1-\alpha$, we require these prediction intervals to satisfy:

where $[\cdot]_{j}$ denotes the $j$-th element of a vector.

We evaluate our prediction intervals based on two criteria: ($i$) Validity: Ensuring the intervals meet the coverage requirements specified in (1); ($ii$) Efficiency: Minimizing the width of the prediction intervals. While wide intervals could trivially achieve the desired coverage, our goal is to produce narrow, informative prediction sets that remain statistically valid and practically useful for decision-making.

We propose a novel method called “Hierarchical Spatio-Temporal Conformal Prediction” (HST-Conformal) based on the conformal prediction (CP) framework [7, 14, 16, 15] to predict the spatio-temporal uncertainty in DER growth. Given a fitted prediction model, the key idea of CP is to construct the uncertainty intervals as a wrapper function around the model’s predictions. The size of the interval is calibrated on a separate calibration dataset through the use of a non-conformity score. which measures the deviance of the model from the designated uncertainty quantification objective. CP is completely distribution-free, meaning that no distributional assumptions need to be imposed as part of the algorithmic procedure, and thus has gained broad popularity in application settings where complex statistical interdependencies may be present between prediction variables.

Specifically, our approach begins by dividing the data into two parts: a training set and a calibration set. The training set is used to create a probabilistic model that simulates future scenarios across all circuits, while the calibration set enables us to evaluate prediction errors and fine-tune prediction intervals for each circuit to ensure valid coverage. One key challenge in this process is the hierarchical constraint of substation-level validity, as described in (1). This constraint has not been considered in prior CP literature, making existing non-conformity scores inadequate for achieving (1). To address this, we design a novel non-conformity score that effectively integrates topology network knowledge, ensuring compatibility with this hierarchical structure. This ensures that the resulting prediction intervals achieve statistical validity without compromising much efficiency compared to prior alternative non-conformity score designs.

We validate our proposed method using a comprehensive real-world dataset of rooftop solar panel installations from a local utility’s network. The dataset spans from 2010 to 2024, consisting of $1,742$ installation records distributed across $n=245$ circuits and $m=51$ substations. This extensive network coverage not only provides an ideal testbed for validating our predictive framework but also enables us to generate actionable insights for system operators. Our analysis delivers valuable decision support for both operational planning and strategic infrastructure management.

In our implementation of HST-Conformal, we set $K=10$, and the parameters of the Hawkes process are randomly initialized, and trained until convergence for $10^{3}$ number of epochs using a learning rate of $10^{-2}$ learning rate with the Adam gradient descent optimizer.

This paper addresses the critical need for multilevel uncertainty quantification in forecasting DER growth by developing a hierarchical spatio-temporal model. Our method jointly predicts circuit-level growth and aggregates them to the substation level, tackling challenges associated with overly conservative prediction intervals and excessive variability in aggregated forecasts. By tailoring non-conformity scores to the unique demands of spatio-temporal data, we ensured that our method achieves both statistical validity and practical relevance across spatial scales. Applied to the local utility’s DER installation data, our model demonstrated improved prediction interval efficiency without sacrificing coverage, offering a robust tool for energy stakeholders navigating the dynamic landscape of DER integration.