A Deep Learning Approach for Macroscopic Energy Consumption Prediction with Microscopic Quality for Electric Vehicles

In order to determine the energy consumption of a metropolitan area under varying conditions, it is necessary to model the entire transportation system. Activity demand is generated to become the basis for the generation of trips. Trips generate traffic flow, which in turn affects average trip speed. Trip information at the link level can then be used along with powertrain information to inform fuel consumption. Autonomie³³3Autonomie is a MATLAB-based software environment and framework for automotive modeling, control system design, simulation, and analysis. ([6] and [7]), POLARIS ([5] and [8]) and SVTrip ([9]) presents a complete workflow to study this complex and multidimensional system, as shown in Figure 2.

POLARIS is used to develop and validate a transportation system model for the Chicago Metropolitan Area. It uses population and vehicle synthesis, calibrated with Census data for the region, along with a complete activity demand generation and scheduling, and subsequent traffic simulation, to model the entire transportation system. On simulating trips in a region, link-level trajectories for all vehicle trips tied with vehicle characteristics is obtained. This route information is fed into SVTrip, which predicts 1-Hz speed profiles for each trip to extract microscopic changes in vehicle movement across a link. The resulting detailed trajectory data is then simulated with Autonomie to estimate the energy consumption of the transportation network for different vehicle technologies. This workflow provides energy estimates in an offline simulation based, high-performance computing setting, and in a one-way (POLARIS $\rightarrow$ SVTrip $\rightarrow$ Autonomie), slow and computationally demanding environment. Because of the high fidelity of the different pieces involved in the workflow, this setup provides accurate energy outcomes. However, it is inadequate for online, on-demand energy calls in which the mobility context is constantly changing, and in which real-time decisions need to be made, such as on-road EV charging decisions and rerouting.

The macroscopic-level trips generated from POLARIS in the workflow in Section II-A contain very basic information (expected average speed over link sequence) about the inner traffic dynamics (Figure 3). The goal is to learn energy consumption on that incomplete summary-level information of the dynamics. Typically, high-resolution speed behavior is key to estimating the energy profile. However, in our setting the learning occurs without it: learn the relationship between high-level route structure and vehicle feature interactions to understand how they correlate with the energy consumed. As a result, in the process described in Figure 2, the high-fidelity time series of vehicle speed dynamics is masked. The data used to train the energy model contain vehicle parameters and high-level aggregated route information only (i.e., macroscopic-level data). This approach is consistent with the information available in real-world situations (e.g., route information from HERE or Google maps). Correspondingly, POLARIS provides trajectory information at the link-level. Generated trips include details about the expected link average speed, link length, speed limits, and similar level parameters. Major vehicle attributes are generally considered available and known; if not, they can generally be retrieved as shown in [10] (vehicle weight, battery size, etc.). Therefore, vehicle information is needed and considered as input to the modeling. Although all internal dynamics that affect energy consumption are masked, we show that it is possible to learn aggregate-level energy consumption values quite accurately with a deep-learning approach. When large-scale data is available, and with some tailored feature engineering, such a model can overcome latent information.

Several scenarios have been defined to cover current and potential future vehicle specifications and transportation behaviors. These scenarios vary in time frame, connectivity and automation, ride sharing, freight systems, and other factors. One scenario reflects today’s situation, where most vehicles are privately owned. Other scenarios represent a short-term future where technology enables people to significantly increase the use of transit, car and ride sharing, and multi-modal travel. Partial automation is introduced, but it is mainly limited to the highway system. Later scenarios represent the long-term future where technology has taken over, enabling high usage of ride sharing and multi-modal trips, which are are convenient and affordable. As a result, private ownership has decreased and e-commerce is common, as is telecommuting. Partially and fully automated vehicles are widely accessible. This extends to scenarios where fully automated vehicles within households are common, and personal ownership results in a low ride-sharing market. The ability to own autonomous vehicles may lead to lower e-commerce and alternative work schedules, and feed urban sprawl. In each scenario the transportation system as a whole is affected, which causes rich and diverse mobility behavior. In addition, with moving time frames, vehicle technologies develop and diversify; in other words, vehicle specifications, power and efficiency maps, component sizes, and other vehicle aspects are affected. This design of simulation creates a dataset that covers a large portion of the sample space. It generates enough examples to model the joint distribution over the vehicle and route features, which allows the subsequent deep-learning model to learn systematic relationships and energy outcomes under a variety of cases. The generated data contains hundred of thousands of EV trips over 30,000+ Chicago links, among more than 3.5 million trips of all types of powertrains (conventional, hybrid vehicles, plug-in hybrids, fuel cells, etc.). It accommodates a wide range of light-duty vehicles, such as compact, midsize, sports utility, pickups, as well as MD/HDT, such as class 3, class 4, class 6, and 8 trucks with different vocations such as pickup and delivery trucks, walk-in, van, box, long-haul, and others. Figure 4 shows a sample count of powertrains and class distribution. As explained before, battery EVs are modeled with varying electric ranges, component sizes, specifications, automation levels (no automation, partial automation, full automation), and other parameters.

Figure 5 shows the distribution and summary statistics of the electric consumption of battery EVs over trips. The electric consumption values are in Wh. We see that the mean is significantly higher than the median, which indicates a large variance of consumption values.

Let $\mathcal{X}=\{(\mathbf{X}_{i},\mathbf{y}_{i})\}_{i=1}^{N}$ be the sample space of data for $N$ trips. $\mathbf{X}_{i}\in\mathcal{R}^{T_{i}\times D}$ is a trip $i$ with $T_{i}$ links and $D$ dimensions so that $\mathbf{X}_{i}=(\mathbf{x}_{1},\cdots,\mathbf{x}_{T_{i}})$, where $\mathbf{x}_{t}=[\mathbf{u}_{t},\mathbf{v}_{t}]\in\mathcal{R}^{D}$ is a link. We define $v_{t}\in\mathcal{R}^{D_{1}}$ and $u_{t}\in\mathcal{R}^{D_{2}}$ as the vehicle features and link features respectively with $D_{1}+D_{2}=D$ dimensional features. The labels $\mathbf{y}_{i}\in\mathcal{R}^{T_{i}}$ represent the electric consumption of vehicles on each link. As a result, the data is constructed as a rank 3 tensor of ($N$ trips, $T$ links, $D$ features). The $D$ features are concatenated from the vectors of vehicle and route features, along with manually feature-engineered ones. As explained before, raw route features data from a macroscopic framework, such as in POLARIS, provide very limited route information. Figure 6 shows the seven main POLARIS link-level activity outputs. These include a link ID, the link entering time for the vehicle, the link length, the expected stop duration on link, the expected link travel time, the expected average speed on link, and the speed limit on link. Because vehicles do not enter or exit the same link at the same time or in the same way, similar (or even identical) links do not necessarily yield similar energy profiles. In a given trip, a route is composed of a sequence of links and the energy consumed will depend on what happened in previous links and what is expected to happen on the next links. In other words, the structure of the current link (the focus) along with the surrounding behavior (the context) will dictate how much energy is expected to be consumed. This a priori known fact embodies our intuition to construct and engineer explicit features to recover a portion of such hidden dynamics (e.g., delta speed of surrounding links). Such macroscopic-level dynamics are the aftereffects of more granular microscopic changes. Designed features that capture sequence dependencies will explicitly guide and inform the model training to learn microscopic latent behavior.

In the dataset used, we have $D_{1}=20$ the dimensionality of the vehicle features, and $D_{2}=26$ the dimensionality of the link features, so that $D=46$. $D_{1}$ includes vehicle data such as vehicle class, weight, battery size, frontal area, etc. $D_{2}$ includes raw route features from macroscopic-level data, in addition to feature engineered ones, such as delta link speeds, delta length, naive congestion calculations comparing average expected speed to speed limits, and other calculated variables.

Our goal is to build a model that can accurately estimate the electric consumption of EVs under large vehicle, roads, trips, and driving variability. Although we have features that seem to directly impact energy consumption, such as trip length, average speed, vehicle weight, there are significant difficulties in modeling the energy consumption explicitly at the link level. It is hard to capture the interactions between links. For example, since the fuel consumption rates for acceleration and slowing down are very different at the same average speed, a vehicle’s energy consumption on a link varies depending on how it enters and leaves the link. A neural network can learn such latent information from a sufficiently large and rich dataset.

In section II-D, we manually and explicitly engineered features based on our knowledge of the problem. We design in this section a deep learning model to implicitly account for link dependencies through recurrent neural network (RNN) sequence modeling. In addition, we pre-process the sequence input with an automated feature extraction mechanism leveraging a convolutional neural network (CNN) ([11], [12], [13], [14]).

More specifically, the fuel consumption of a vehicle on a link depends on the preceding and following links. For that modeling portion we leverage a bi-directional RNN with long short-term memory (LSTM) cells [15] stacked with a fully connected neural network as a series of time-distributed dense layers of varying dimensions. The time-distributed property preserves the output sequence for link-level evaluation of electric consumption (figure 7). The bi-directional aspect (BiLSTM, [16]) captures the contextual qualities surrounding the current link. BiLSTMs understand context better because the sequence runs forward and backward, and two internal states summarizing the entire trip are combined. Overall, this portion of the network attempts to learn the sequential dependencies of links and their consumption. A high-level overview of the model is that, given an input batch of trips $\mathbf{X}$ with length $\tilde{T}$, the output is formulated as

where for an $h$ the size of the hidden units of the BiLSTM stack, $\mathbf{H}\in\mathcal{R}^{\tilde{T}\times h}$, $w_{rnn}\in\mathcal{R}^{h}$, and the output $\tilde{y}_{rnn}\in R^{\tilde{T}}$. The $g(\cdot)$ is the rectified linear unit (ReLu) activation function, which computes $f:x\mapsto max(0,x)$.
While the LSTM captures the context between links via message passing of computed internal cell states (capturing sequential dependencies), the CNN implicitly searches for direct local structures and interaction between the links as a feature extractor mechanism. The combination of CNN and LSTM allows us to enrich the feature space and expand the receptive field of trips over the links.

Our goal is to estimate electric consumption at the link level while ensuring trip-energy accuracy is preserved, that is $\hat{y}^{i}_{trip}=\sum_{t=1}^{\tilde{T}}\hat{y}_{t,i}$. A new trip can be a new combination of existing (adjacent) links and unknown (i.e previously unseen) to our model. Therefore, we require our model to be accurate on both link-level and trip-level consumption. Meanwhile, to further improve our model’s generalization ability, we impose an additional cumulative link loss that requires the model to be precise on partial trips up to each link. Our loss is formulated as:

We search the optimal solution,

through back-propagation gradient descent, where,

evaluates the model’s performance on the partial trip up to the $l$-th link. We sum $C_{l}$ over all $l$ to be the cumulative link loss,

which evaluates the model’s predictions on partial trips up to each link. The trip-level loss

evaluates the model’s prediction on the final consumption of the entire trip, and

measures the model’s performance on each link. As mentioned in the previous section, we group trips based on their length without padding. Trips in the same batch share a common $T_{i}$, so the objective function (2) is precise in training.

The sequence modeling aspect of the model is wrapped in an encoder-decoder architecture [17]. This allows the model to learn whole trip representations using an RNN encoder block. The encoder summarizes the entire trip into a fixed-length vector $c$ representing the entire sequence. The decoder is a second RNN block that is trained to generate the electric consumption outputs at each link. In the decoder, $y_{link,i}$ and $h_{link,i}$ for link $i$ are both conditioned on $c$, but also on the previous and next link states and inputs, via the bidirectional passes. The encoder and decoder are jointly trained to minimize the loss function in equation (2). In our implementation, our encoder RNN is a deep LSTM with 2 layers, and 512 units at each layer. The decoder is a bidirectional LSTM of 256 units initialized with the encoder hidden and cell states. We stack the decoder with several fully connected neural network layers of size 128, 64, 32, and the last layer has 1 unit. As explained before, the neural network layers are time distributed; in other words, the same weights are applied to the encoder output for each link.

The input data is pre-processed before being fed to the encoder-decoder, by a 3-layer deep convolution neural network with residual connections between the layers. Batch normalization and max-pooling are performed at each layer. Kernel size was fixed at 3, while the number of filters doubled at every layer, starting with 32 filters. The convolution neural network was leveraged as a feature extractor of important trip features while also shortening the sequence at every pass of a convolution block. See Figure 8 for an overview of the architecture.

Other implementation details:

• •

  We initialized all of the LSTM’s parameters with the uniform distribution. Non-recurrent weights were initialized by sampling from the isotropic zero-mean (white) Gaussian distribution.

• •

  Optimization is performed with stochastic gradient descent without momentum. We employed a dynamic learning rate, initialized at 0.001 and decay proportional to $epoch^{-1/2}$.

• •

  Due to bucketing of trips (as per Section II-C), the batch size is variable but capped at 128 trips. The bucketing speeds up processing because batches contain similar sequence length and no computation is wasted on the presence of potentially shorter sequences.

• •

  Standard ReLu activation functions were used for convolutional and dense layers, and $tanh$ was used for recurrent layers.

• •

  Excluding the convolution and recurrent layers, we used dropout regularization.

For training and tuning purposes, we leveraged multiple graphical processing units (GPUs) spread across multiple machines with a multi-worker mirrored distributed training strategy. This strategy implements synchronous distributed training across multiple workers, each with potentially multiple GPUs. With the help of roughly 30 machines, each with at least 1 GPU, we were able to decrease the model training and tuning time for a dataset of this scale. Our hyperparameter search space contained different hyperparameters spanning different areas of model development including the different pre-processing approaches discussed in Section II-C, model selection and design, and learning rates. In the remaining sections, we present the results of the best trained model.

We evaluate the model performance on root mean square error (RMSE) and mean absolute error (MAE) at the link and trip levels. However, there are some long trips in the dataset where the energy consumption accumulates to tens of thousands of Wh. In these cases, a 1% error in prediction will lead to a large-magnitude RMSE and MAE. Therefore, we also evaluate our model on mean arctangent absolute percentage error (MAAPE). MAAPE is less sensitive to the standard mean absolute percentage error [18], especially when $y^{i}_{\text{trip}}$ is small.
MAAPE is formulated as:

It averages the AAPE, $\arctan\Big{|}\frac{y^{i}_{\text{trip}}-\hat{y}^{i}_{\text{trip}}}{y^{i}_{\text{trip}}}\Big{|}$ where each absolute percentage error is bounded by $\frac{\pi}{2}$ in MAAPE, so the value of MAAPE will not be significantly affected by a small number of outliers when the sample size is large. Arctangent absolute percentage error also approximates absolute percentage error well for small errors. From a Taylor series of $\arctan{x}$, we see that $|\arctan{x}-x|=\mathcal{O}(\epsilon^{3})$ for $x\in[0,\epsilon]$. As a result, MAAPE can give us a better idea of the overall performance of the model when there are outliers in errors.