Edge-Cloud Collaborative Satellite Image Analysis for Efficient Man-Made Structure Recognition

Edge computing has emerged in recent years and is being applied to various fields, representing a significant shift in data processing pattern, positioning itself at the frontier of network architecture[9]. This innovative approach involves analyzing and processing data at the periphery of the network, or ”the edge,” where data is captured and lightweight models are deployed. Unlike traditional centralized computing pattern, which relies on data being sent to remote servers for processing, edge computing brings computational resources closer to the data sources—be it IoT devices, industrial machinery, or mobile devices. This proximity reduces the latency and bandwidth issues often associated with data transfer to central servers, allowing for near-instantaneous data processing and bringing many benefits.

In the domain of autonomous vehicles, a study by Lv et al.[10] examines the integration of edge computing with 6G networks to enhance environmental perception capabilities of autonomous vehicles. In industrial automation, a compelling study by Schmidt et al.[11] describes the development of a low-cost, Raspberry Pi-based automation system for hydraulic processes, embodying principles of Industry 4.0. In healthcare, an innovative study by Rajavel et al.[12] presents the IoT-based smart healthcare video surveillance system, enhanced through edge computing. This system, named the Cloud-based Object Tracking and Behavior Identification System (COTBIS), dramatically improves fall detection accuracy and reduces response times by implementing advanced video processing algorithms at the network edge.

When applied to satellite image, edge computing differentiate itself from traditional computational methods by addressing unique challenges posed by the satellite environment. In satellite systems, particularly those in Low Earth Orbit (LEO), data transmission to ground stations is often hampered by latency, intermittent connections, and limited bandwidth, Moreover, satellites inherently possess constrained computing power and energy resources, necessitating highly efficient data processing solutions that minimize power consumption and computational load.

One innovative application of edge computing in satellites is the work by Trong-An Bui et al.[13], which leverages an edge-computing-enabled deep learning model for enhancing low-light image processing on satellite platforms. This model employs a lightweight encoder-decoder architecture designed to perform under the constrained computational resources available on satellites. Yijie Chen[14]investigates how to optimize the energy and computational demands that burden traditional DNN applications in satellite operations. Chen proposes a method where DNN tasks are strategically segmented and distributed between satellite platforms and ground stations. This approach not only minimizes latency and power consumption but also enhances the overall efficiency of data processing, making real-time analysis and decision-making feasible even in complex satellite network scenarios. Ming Zhao[15] highlights the integration of collaborative machine learning with satellite operations to enhance the intelligence and responsiveness of satellite systems. Zhao’s framework uses asynchronous learning algorithms to accommodate the dynamic and delay-sensitive nature of satellite communication. Bradley Denby[16] explores the transformative potential of Orbital Edge Computing (OEC) within nanosatellite constellations for remote sensing applications. Employing advanced computational modules like the Jetson TX2, Denby’s work showcases how edge computing can be implemented directly on satellites to perform sophisticated machine inference tasks onboard. This capability is particularly beneficial for applications requiring immediate data processing, such as environmental monitoring and crisis response, where the speed of data acquisition and analysis is critical. The OEC framework significantly reduces the dependency on ground-based processing infrastructure, thereby decreasing the latency and increasing the timeliness of satellite data utilization.

The UCMerced Land Use Dataset was used for this paper. The dataset consists of 2100 aerial scene images, which are categorized into 21 land use classes. The 21 classes include a variety of natural or human-made scenes such as agricultural lands, forest, golf courses, overpasses, parking lots, and residential areas. Each class contains 100 images, offering a balanced dataset that facilitates comprehensive training and testing of machine learning models. The images are uniformly sized at 256x256 pixels, captured at a resolution of 30 centimeters, providing high-detail visual information that is suitable for fine-grained classification tasks.

In this study, we propose a hybrid edge-cloud collaborative architecture for efficient recognition and classification of human-made structures in satellite imagery, as shown in Fig 1. Our methodology consists of initially training a sophisticated model on a local machine using high-performance computing resources. This model focuses on the preliminary identification of images containing man-made structures amidst natural landscapes. A lightweight model is deployed at the edge to handle incoming satellite images, facilitating real-time data processing and immediate decision-making.

At the edge, a lightweight model processes this dataset to swiftly segregate natural landscapes from potential man-made structures, thereby streamlining the dataset to focus solely on images relevant to our task. This filtration significantly alleviates the computational burden and reduces the bandwidth required for transmitting data to the cloud. Once the edge system identifies images containing man-made structures, these are forwarded to the cloud for further analysis. On the cloud, a more sophisticated and computationally intensive model undertakes the task of detailed classification, discerning specific types of man-made structures such as harbor, runway, or sparse residential area. This stage leverages the cloud’s enhanced processing power to conduct a thorough and precise analysis, although it incurs a higher time cost due to the depth of processing involved. This hybrid processing framework optimizes the use of distributed computing resources, effectively balancing speed and accuracy in the classification of complex images. The classification of specific types of human-made structures is an important component of the pipeline; however, it falls outside the scope of this paper and will not be addressed in the subsequent sections.

Various combinations of models were deployed on edge to explore the optimal performance. Neumann, Maxim et al[17] achieved an accuracy of 0.9961 on UCMerced Dataset using ResNet50. MobileNet[18] and ShuffleNet[19] provide an efficient and effective framework for running deep learning models on mobile and edge devices. For every model, we conducted 10 experiments and take the average to avoid the interruption of random netspeed.

Here’s a brief introduction about the models we tried in the experiment. SimpleCNN is a self-built convolutional neural network featuring three convolutional layers. Each layer is followed by batch normalization and max pooling to enhance model stability and reduce overfitting. MobileNetV2 is designed with specific adjustments to its layer configurations and structures, optimizing performance while reducing computational demands and improving accuracy for targeted tasks. We tried ShuffleNet, an efficient convolutional neural network known for its use of pointwise group convolutions and channel shuffle operations. These features significantly reduce the model’s complexity and computational cost, making it ideal for applications where resources are limited, without compromising on accuracy.

The proposed model of this paper, MobileShuffleNet (MSNet, Fig 2), is an advanced, compact deep learning architecture, particularly suitable for scenarios with limited processing capabilities like handheld devices or on-the-edge computing platforms. In designing the hybrid network architecture, our primary objective was to harness the strengths of both MobileNetV2 and ShuffleNet to create a network that is both lightweight and maintains high performance. MobileNetV2 is renowned for its efficient architecture that employs depthwise separable convolutions to significantly reduce the model’s parameter count and computational complexity. On the other hand, ShuffleNet optimizes performance through the use of grouped convolutions and channel shuffling, enhancing both efficiency and information flow.

Initially, the model utilizes the initial layers of MobileNetV2 for foundational feature extraction, predominantly leveraging depthwise separable convolutions for efficient feature processing. Depthwise separable convolutions reduce computational cost by separating the convolution into a depthwise spatial convolution and a pointwise convolution, which can be formally expressed as:

where $X$ is the input, $k_{k}$ represents the channel-wise convolution kernels, $K$ is the $1*1$ convolution kernel, $Y_{k}$ denotes the output of the depthwise convolution, and $Z$ is the final output. This approach drastically cuts down the number of parameters and the computational burden.

Subsequently, the model incorporates key technologies from ShuffleNet, namely grouped convolutions and channel shuffling. Grouped convolutions divide the input channels into multiple groups, each performing convolutions independently, which not only reduces the parameters further but also lessens the computational load. Channel shuffling ensures effective inter-group information exchange to prevent the isolation of information within groups. Specifically, after each group convolution, a permutation operation rearranges the output feature maps to maintain the network’s expressive capability and generalization.The mathematical representation of grouped convolutions can be described as follows:

where $X$ is the input feature map, $W_{i}$ represents the convolutional filters for the i-th group, and $G$ is the number of groups.

Channel shuffling can be mathematically expressed as:

where $G_{i}$,…,$G_{G}$ are the grouped convolution outputs before shuffling, and $Y$ is the shuffled output which combines features from each group.

To further refine the feature recombination process, a permutation operation rearranges the output feature maps between grouped convolutions:

where $R$ denotes the recombined features, $W_{i}$ represents the weights applied to the features of each group $i$, $Y_{i}$ is the output from the channel shuffling phase for each group, and $\sigma$ is an activation function enhancing non-linearity.

Finally, the network concludes with a simplified classifier comprising global average pooling followed by a fully connected layer. Global average pooling serves as a feature compressor, helping to mitigate overfitting and reduce computational demands, while the fully connected layer caters to task-specific classification. The entire model is designed to strike a balance between computational efficiency and performance, making it suitable for applications that require rapid processing under constrained resources.

The experiment is simulated using Ubuntu22.04 virtual environment. Images are resized to a uniform dimension of 64x64 pixels and normalized. The UCMerced dataset is categorized into ”Artificial” and ”Natural” groups. Artificial classes include structures such as dense residential areas, storage tanks and more. In contrast, natural classes include samples labeled as forests, rivers and others. In this experiment, cross-entropy loss and the Adam optimizer were selected to adjust the model’s weights. The training consisted of 25 epochs, with an initial learning rate set at 0.001.

We meticulously assess both the accuracy and latency of our edge-cloud collaborative model to discern its effectiveness in processing satellite imagery for the detection of man-made structures. Our evaluation is structured to provide a holistic view of the model’s performance under operational conditions, focusing on two primary metrics: accuracy and latency.

In our evaluation of latency, we dissect the time taken at two critical stages to ensure a comprehensive understanding of the system’s performance. The first stage, edge processing time, refers to the time required by the satellite’s onboard processing system to conduct initial image analysis and classification. This includes any preliminary computations such as feature extraction and basic classification that occur before the data is sent to Earth. The next phase, Transmission Time, is the period needed to transmit the processed data or specific segments of interest from the satellite back to ground stations. This stage is vital as it directly affects how quickly the data becomes available for subsequent analysis, impacting the overall responsiveness of the system. We simulated the weak internet environment typically encountered in space by introducing obstacles to data transmission within an Ubuntu 22.04 virtual machine. Each of these procedures collectively contributes to the total latency of the system, and understanding these components helps in optimizing the pipeline for quicker and more efficient satellite data processing.

The results of the experiment are detailed in Fig 3 and Table 1. Fig 4 and Fig 5 illustrate examples of the prediction alongside the true label. In this study, the edge processing time is influenced by the model’s characteristics, while the transmission time depends on the number of images transmitted, a factor also determined by the model. Two key evaluation metrics of interest are F1-score and total time. We found that the proposed method significantly outperforms others. MSNet reduces edge processing time by approximately 37% compared to ShuffleNet (from 1.02 seconds to 0.64 seconds). The total time required by MSNet is reduced by 11% compared to ShuffleNet (from 3.67 seconds to 3.25 seconds). When it comes to accuracy, MSNet remains a high F1-score of 0.97, basically same as MobileNetV2 and a bit higher than ShuffleNet.

However, as shown in Fig 4, although MSNet achieves a high F1-score, there are still some incorrect predictions. These errors could lead to severe consequences in practical applications due to their potential for critical oversights. Further studies could be conducted to mitigate these issues.