Multi-task Learning for Sparse Traffic Forecasting

First, the data records are clustered into 10 groups following the simple median clustering method¹¹1https://github.com/iarai/NeurIPS2022-traffic4cast/ provided by the organizer²²2https://www.iarai.ac.at/traffic4cast/. Specifically, for each data record, we add the values of all loop counter volumes to obtain a volumeSum value. Then, we first sort the dataset according to volumeSum and then patition the records into $\displaystyle 10$ equal-frequency bins. The bin number indicates the cluster index of each data record.

Afterwards, for each road segment, we extract a $\displaystyle 10\times 3$ feature matrix $\displaystyle V$ as follows. Suppose there are $\displaystyle n$ historical records within the $\displaystyle i$th cluster, while for this road segment, the numbers of congestion labels for the undefined/green, yellow, and red states are $\displaystyle c_{i}^{1}$, $\displaystyle c_{i}^{2}$, and $\displaystyle c_{i}^{3}$, respectively. Then the $\displaystyle i$th row of $\displaystyle V$ is calculated as $\displaystyle(c_{i}^{1},c_{i}^{2},c_{i}^{3})/n$. In this way, we obtain the statistical distribution of the congestion level of each road segment w.r.t. different global traffic volumes, which are taken as the prior knowledge and passed into the graph neural network for further learning.

In our road graph, loop counters are very sparse. If the volumes of loop counters are directly introduced into the network as node features, there will be many vacant values in nodes. So we use a multi-layer perception network to learn the relationship between counter volume and road segments, and then feed the feature of road segments to the graph neural network for further learning.

The attribute features of road segments are essential. We embed importance to $\displaystyle\boldsymbol{R}^{5}$, oneway to $\displaystyle\boldsymbol{R}^{2}$, tunnel to $\displaystyle\boldsymbol{R}^{2}$, and lanes to $\displaystyle\boldsymbol{R}^{3}$. For continuous features like parsed max speed, flow speed, length meters, counter distance, and limit speed we just concatenate them. By this way, We obtain a unified representation for each road segments.

GNN has achieved good results in many fields, it can aggregate information of other nodes and edges through a message passing mechanism. Notice that the loop counter volume data is sparse, which means the nodes are hard to learn the features of their neighbors by normal graph neural network. Moreover, our tasks aim to learn the features of road segments, but the GNN model aggregate information based on nodes. So it is more conducive to the representation learning of road segment features by taking road segments as the nodes of the graph. For this reason, we construct a new road graph, taking road segments as the nodes of the graph. In this way, we can better learn the dependencies between road segments. We introduce the features learned by the previous modules into the multi-layer graph neural network, learn the spatial dependency relationship between road segments, and pass the output to the multi-task learning network.

We propose a multi-task learning component to the learn congestion class, the speed value, and the volume class for each segment in the road graph in a concurrent way. We feed the output of GNN module to residual blocks [16] to obtain the congestion class $\displaystyle\boldsymbol{y}_{c}^{pred}$, speed $\displaystyle\boldsymbol{y}_{s}^{pred}$, volume class $\displaystyle\boldsymbol{y}_{v}^{pred}$. The loss function is defined as follows:

Where $\displaystyle\boldsymbol{y}_{c}$ denotes the label of the congestion class (0=undefined; 1=green/uncongested; 2=yellow/slowed-down; 3=red/congested), $\displaystyle\boldsymbol{y}_{s}$ denotes the label of the speed, and $\displaystyle\boldsymbol{y}_{v}$ denotes the label of the volume class (1 for volumes 1 and 2; 3 for volumes 3 and 4; 5 for volumes 5 and above, according to the competition).

Finally, we combine the loss terms of these tasks as the overall loss function:

where $\displaystyle\lambda_{1}$, $\displaystyle\lambda_{2}$, and $\displaystyle\lambda_{3}$ are hyper-parameters. We take 0.03, 1, 1 in training.

Since we have the speed output and length of all road segments, we just need to calculate the travel time of segments in the super-segments and add them all together to produce the super-segment ETAs.

For each city dataset, we trained 9 models, with each model taking 20 rounds of training in order to select the best-performed one on the validation set. Then, we average the predicted results of the 9 model to produce the final submission result. Finally, our team GongLab won the fourth place in the core competition and the sixth place in the extended competition.

We compared our model against several baselines on the validation set:

• •

  Naive Count: For the core challenge, we count the congestion category of all roads and calculate the probability. For the extended challenge, we calculate the median eta of all super-segments in the training set.

• •

  Volume Cluster: The baseline groups the data based on the sum of counter volumes. It calculates the statistical probability of congestion for different groups for the core challenge and the median ETA for different groups for the extended challenge.

• •

  GNN: The GNN model is provided by the organizers. It fills the loop counter volumes of nodes as input and uses message passing mechanism to learn the dependence between nodes.

As can be seen from Table 5 and Table 6, our model performance is significantly better than the baselines. The Naive Count model does not take the dynamic changes of input into account, and Volume Cluster model cannot learn the dependence between road segments. Since the loop counter volumes are very sparse, the performance of the GNN model is poor. Our model combines the Volume Cluster and GNN methods, which can capture the dynamics of volume data as well as the dependencies between road segments. Moreover, we adopt the multi-task learning method to learn the congestion class, the speed value, and the volume class simultaneously, which further boosts the performance.

We perform ablation experiments to verify the effectiveness of the model. It can be seen from Table 7 that the performance reduces after removing each module we proposed, which shows that the above modules are meaningful. For example, if the clustering component is removed, the score of the model increases from 0.82705 to 0.84949, indicating that it is important to use volume clustering method to capture the dynamic changes of loop counters data.