FLEE-GNN: A Federated Learning System for Edge-Enhanced Graph Neural Network in Analyzing Geospatial Resilience of Multicommodity Food Flows

In this section, we introduce the components of FLEE-GNN, which include a message-passing-based GNN and a specially designed federated learning system to handle cases where there is no freight transportation or information exchange between geographical regions (“hypothesized disruptions”) in the U.S.

The U.S. Bureau of Transportation (BTS) and the U.S. Census Bureau publish the detailed commodity food flow survey (CFS) ¹¹1https://www.bts.gov/product/commodity-flow-survey once every five years, which presents a data scarcity challenge for the idea of measuring food supply resilience using deep learning methods. To overcome this challenge, we introduce an adjustable data generator (Algorithm 1), in conjunction with the previous entropy-based method (Rao et al., 2022), to create nearly authentic food flow data. This data generator requires a real commodity flow dataset as input, accompanied by a noise ratio. The noise ratio is defined as the proportion of data we wish to modify based on the actual dataset. We have also delineated three methods to adjust the data:

1. (1)

   ADD (Algorithm 2): Introduce a transportation event for food type $c^{\prime}$ from source $s^{\prime}$ to destination $d^{\prime}$ with a random value $v^{\prime}$, tonnage $t^{\prime}$, and average transportation miles $a^{\prime}$ sampled from its corresponding distribution in the original dataset. If the sampled tuple $(s^{\prime},d^{\prime},c^{\prime})$ is already present, we sample another one.

2. (2)

   REMOVE (Algorithm 3): Randomly select an edge and expunge it from the current food flow.

3. (3)

   CHANGE (Algorithm 4): Randomly select an edge and resample value $v^{\prime}$, tonnage $t^{\prime}$, and average transportation miles $a^{\prime}$ from its corresponding distribution in the original dataset.

Given a noise ratio, we evenly apply the three aforementioned operations to generate a new food flow dataset for data augmentation purpose. For instance, if the original datasets contain 100 edges and we have a noise ratio of 0.3, we will add 10 edges, remove 10 edges, and change 10 edges. This results in a dataset with 100 edges, albeit different from the original set.

In our experiments, we utilize the above-mentioned U.S. CFS data. The CFS provides comprehensive information of domestic freight shipments including commodity type, value, weight, distance shipped, origin and destination, etc. from national-level to state-level. As an example, we focus on the agricultural multi-commodity flows in 2012 and 2017 at differential geographical regions by extracting the data with SCTG code from 01 to 08. We further process the raw data into two primary datasets:

1. (1)

   Silo Dataset: Assumes that each silo (i.e., geographic region) can only access the links and transitions within its domain.

2. (2)

   Centralized Dataset: Encompasses all links and transitions, both within and across silos, thus serving as a comprehensive graph of transitions.

The basic graph statistics of the whole food supply network and different sub-networks by regions are illustrated in Table 1 and Table 2. The sub-networks of food flows in different geographical regions have distinctive graph statistics (e.g., degree, centrality, connectivity), which reflects spatial heterogeneity under different spatial configurations and poses challenges on geospatial resilience modeling and evaluation.

The baseline model, derived from our previous work (Rao et al., 2022), employs an entropy-based method using the centralized food flow dataset. Given its access to all links between geographical regions, we designate this model as our ground truth.

A node-level resilience $R_{i}$ that comprehensively measures the single-supplier/customer dependence, single-commodity-type dependence, transport distance, and geographic adjacency of the node $i$ is computed as:

where $D_{i}$ is the overall single-commodity-type dependence of node $i$ (measured by the Shannon information entropy); $Agg$ is a set of aggregated commodity types; $V^{\prime}_{(i,A)}$ is the value of aggregated commodity $A$ reflecting single-commodity-type dependence; $V^{\prime}_{i}=\sum_{A\in Agg}\sum_{c\in A}\sum_{j}V^{\prime}_{(i\rightarrow j,c)}$ denotes the total commodity value of node $i$ combined with average transport miles and geography adjacency information. The higher the $R_{i}$, the less dependent node $i$ is on single or geographically distant/non-adjacent supplier/customer or single commodity type, thus the higher the resilience.

We set the developed FLEE-GNN model against the entropy-based method and a centralized GNN (CT-GNN). We analyzed how each model performed under various noise levels by perturbing the 2012 food flow data at noise ratios of 0.1, 0.3, and 0.5, resulting in three datasets of 500 graphs each. Both CT-GNN and FLEE-GNN models underwent training for 100 epochs. Evaluations were then carried out on real food flow data from 2012 and 2017, respectively.

We define the relative resilience difference as the difference between each model output and the entropy-based computation result (ground truth). As seen in Figure 3, it is evident that the resilience values generated by the entropy-based method within sub-networks only take into account data within a silo, causing the model to underestimate its global resilience in the food supply network. This can be attributed to the omission of transportation between different geographical silos. By neglecting these transportation movements, we effectively remove edges from the original spatial network. As a result, nodes, especially those that lose transportation connections, become more vulnerable to potential disruptive changes imposed on the food systems.

In centralized training (Figure 4), the full graph is used both for training and evaluation. Consequently, there isn’t any significant overestimation or underestimation observable from the graph. In the case of FLEE-GNN (Figure 5), the silo sub-networks and ground truth label from the full graph are used for training. This simulates the scenario where we only have access to silo data in the real world due to privacy constraints. Yet, from that silo data, we still aim to recover the original true resilience of the full graph. During the training, our model tends to overestimate its resilience. However, this overestimation is offset by the lack of information in the actual global graph during evaluation. Through such a strategy, we also achieve balanced predictions when using FLEE-GNN without significant overestimation or underestimation.

While the relative resilience difference captures the proportionality of errors, highlighting under- or over-estimations, we also examine the absolute prediction error, which quantifies the absolute deviations. For each model evaluated on data from both 2012 (Table 3) and 2017 (Table 4), we calculated the mean, standard deviation, minimum, 25th percentile, median (50th percentile), 75th percentile, and maximum values. The results indicate that both centralized training CT-GNN and federated training FLEE-GNN (with smaller prediction errors), trained on datasets generated with varying noise ratios, outperformed the entropy-based method across all metrics.

Another trend we observed from the results is that the performance tends to degrade slightly as we increase the noise ratio. This can be attributed to the fact that by injecting more noise, the generated food flow data becomes less representative of real data. Modern machine learning techniques often encounter this out-of-distribution issue (Yang et al., 2021). Furthermore, when comparing results between 2012 and 2017, the model maintains its performance even when evaluated on the 2017 food flow data. This is notable since the training data comes solely from the 2012 food flow dataset. Despite this, the model still performs admirably in predicting resilience for the food flow network in a different time period, demonstrating its versatility and effectiveness.

Rather than focusing solely on the specific resilience provided by the model, we are also interested in how the rankings of node resilience in food supply networks are maintained. Specifically, we aim to determine whether a state with high or low resilience will consistently show high or low resilience values in comparison to other states in the FLEE-GNN model’s output. To assess the consistency between predicted and actual state rankings based on their resilience values, we sorted the states accordingly and computed the following concordance metrics:

1. (1)

   The coincidence rate between the top n% ranked nodes of actual data and the top n% ranked nodes of predicted data.

2. (2)

   Pearson’s correlation coefficient.

3. (3)

   Spearman’s rank correlation coefficient.

From the results (Table 5 and Table 6), it is evident that the machine learning-based method also boasts a superior coincidence rate at different noise levels of data, signifying that these models offer higher consistency and alignment with the actual resilience values in the original dataset. Additionally, a higher Spearman’s rho indicates that the rankings from the predictions of CT-GNN and FLEE-GNN closely match the actual rankings. Likewise, a larger Pearson’s R indicates a robust linear relationship between predicted and actual resilience values. This means that as the actual resilience values rise (or fall), the model’s predictions will also increase (or decrease) in a similar fashion.

Not only are we interested in the performance of FLEE-GNN at different time periods (2012/2017) and in different spaces (global transportation, silo transportation), but we are also keen on its performance across different missing data scenarios. In these experiments, we compare the absolute prediction error for CT-GNN trained with data generated from the 2012 food flow network and evaluated on the 2017 global network data, with that of FLEE-GNN trained with the same data and evaluated on 2017 sub-network silo data. However, during training, we consider different data combinations to simulate scenarios where certain edge features missing from a dataset.

• •

  V: Represents that the value of commodity food flow exists in edge features.

• •

  A: Represents that the average transportation mile of commodity food flow exists in edge features.

• •

  T: Represents that the tonnage mile of commodity food flow exists in edge features.

For example, ”AT” means the graph structure of the food supply network remains the same, but instead of having access to all the shipment value, average transportation mile, and tonnage, we lose the shipment value information.

As seen in Table 7, interestingly, both CT-GNN and FLEE-GNN maintain stability even with some missing edge features. We do notice that some performance decreases when the data constrain the type of information the model can access. The maximum difference in terms of mean absolute prediction error for CT-GNN is $0.0793-0.0599=0.0195$ and for FLEE-GNN is $0.0978-0.0808=0.017$.

To some extent, this research proves that the most relevant information supporting current resilience measurement is embedded in the graph structure. In the extreme NONE case (in Table 7), all we retain is the basic graph structure (topological relations between nodes) without any edge features. Still, both CT-GNN and FLEE-GNN provide a satisfactory estimation of the resilience value. However, the original entropy-based method, which relies heavily on the edge attribution, fails to generate a meaningful resilience score in this case.

The silo-data-based resilience metrics using the entropy method revealed its constraints, particularly in its limited access to comprehensive linkages among geographical regions. While both the CT-GNN and FLEE-GNN demonstrated performance robustness against noise, and FLEE-GNN outperformed consistently, especially in higher noise scenarios. The Spearman rank correlation of the FLEE-GNN indicates a strong agreement with the ground-truth data, emphasizing the model’s utility in accurately ranking the resilience of U.S. states in food supply. Given that the eventual aim is to bolster food security, a model like FLEE-GNN offers valuable insights into spatial network vulnerabilities and opportunities to improve supply network resilience. This could guide food systems policy-making, resource allocation and negotiation in critical scenarios in advance of cascading system failures.

In summary, FLEE-GNN not only holds promise as an innovative tool for assessing food network resilience but also sets the stage for leveraging federated learning in a myriad of complex, geospatial AI applications. Future directions might delve deeper into optimizing FLEE-GNN and expanding its applicability to other domains.

Our initial work utilized labels generated by the entropy-based method as the ground truth. However, this limits us to the upper bound provided by the entropy-based method. Unsupervised learning has demonstrated greate success in the field of image classification (Schmarje et al., 2021), feature selection (Khanum et al., 2015), association rule detection (Reshef et al., 2011), etc. Developing unsupervised learning techniques may allow us to exceed this bound and improve our system’s predictive power.

We currently input different attributes into the FLEE-GNN as scalar values. The model thus lacks an understanding of the differences between values, tons, average miles, temperature control, and other attributes in food supply transport. Moreover, it cannot extract comprehensive relationships between these attributes. To improve our model’s predictive ability and interpretability, we need to devise a better explainable AI method for embedding GeoKG information directly into our system.

Moreover, the emphasis on federated learning within FLEE-GNN opens doors for its application in scenarios where data privacy and decentralization are paramount. This consideration will undoubtedly grow more significant in an era marked by heightened data protection regulations and the need for more inclusive, globally-spread research collaborations.

In conclusion, while the immediate implications of our work pertain to food supply networks, the FLEE-GNN framework offers a versatile blueprint with applications across a multitude of sectors. We eagerly anticipate the breakthroughs and innovations that can arise as we venture into other types of spatial networks and application domains.