Satellite Detection in Unresolved Space Imagery for Space Domain Awareness Using Neural Networks

Classification and detection of satellites in space imagery is important for various use cases such as safety, reconnaissance, contingency planning, space, and debris removal. With the increasing number of satellites in orbit especially due to new super constellations such as SpaceX’s StarLink, and OneWeb, rapid and efficient methods of processing space situational awareness (SSA) observations must be developed. Current estimates put around 700,000 objects larger than 1cm in orbit, the vast majority consisting of uncontrolled debris.[1, 2] Various methods have been used for processing optical observations of resident space objects (RSOs). Some methods include, gross motion of objects across frames, inertial frame angular observations, and template matching.[3, 4, 5, 6, 7, 8] The core of any Space Domain Awareness (SDA) or SSA algorithm involves finding the camera frame location of an RSO in an image and segmenting the RSO from the background star-field. The detection and classification of satellites, space stations, and/or space debris using Computer Vision (CV), Machine Learning (ML), and Artificial Intelligence (AI) algorithms has recently spurred interest in space imagery. Some methods rely on different CNN architectures and Phyiscs Informed Neural Networks (PINNs). Fletcher et. al. used a Convolutional Neural Network (CNN) to detect Geosynchronous objects in unresolved space imagery.[9] And it was shown that the algorithm outperformed the classical SExtractor object segmentation algorithm on a large database of imagery of Geostationary (GEO) satellites.[9] Kyono and Fletcher also studied resolved imagery, and partially resolved imagery to estimate the quality of optical images of satellites.[10] Peng proposed a method of using Gaussian processes to augment orbit determination methods by learning the offset from lower fidelity predictions to true observations, and is applicable to the orbit determination steps to be considered in the future.[11, 12] Another study looked at the possibility of using imagery from star-trackers for SSA purposes and used a Recurrent-Convolutional Neural Network (R-CNN) to estimate RSO position and velocities.[13] Small segments of each image were passed through a CNN for object detection, and then the Recurrent Neural Network (RNN) was used to detect the sequence of observations of an RSO; however, all images in the database were synthetic.[13] Woodward, extended the work of Fletcher and trained a CNN for pixelwise segmentation and detection of RSOs using the same SatNet database from ref. [9, 14]

This work is aimed to generate a database of unresolved space imagery containing both images with and without RSOs, train a MobileNetV2 CNN using the custom database, validate the CNN on external images set, assess its performance, and finally discuss future work where it is planned to build upon this CNN framework to extract the orbital elements set for orbit determination using novel Product of Exponentials (PoE) orbital mechanics framework [15]. The MobileNetV2 CNN architecture was chosen for satellite detection because it has a fewer number of parameters and is faster than most of the alternative models [16, 17]. Furthermore, MobileNetV2 does not require considerable computational resources to train and infer, which makes it suitable for on-orbit training and detection. As a result, a goal for this work was training sample efficiency and performance on a limited dataset to simulate such conditions. This CNN model design can be utilized for a variety of tasks, including segmentation, but it was employed for object detection in this work.

As the mobile deployment of autonomous satellite tracking is a significant focus for this work, the minimization of model inference times on the Raspberry Pi was a crucial factor. Keeping this in mind, conversion to TFLite allows for the limited resource availability while keeping inference latency low. The scenarios for tracking satellites (either as small streaks or as point-source dots) were tested with MobileNetV2 and the two versions of EfficientDetLite as well. These models, while trained solely on the SatSim-generated images, were given portions of the real and synthetic satellite image databases for inference testing and verification. Results from the MobileNetV2 model are shown in Figure 4 where confidence levels and RSO detections between the TFlite and TF versions are paired with mixed imagery. For comparison, the inference performance from the EfficientDetLite models can be seen in Figure 5.

It is worth noting that inference on the two tracking modes requires separate models. This is due to the nature of the target bodies shape profiles completely switching places when changing tracking reference. In early testing, tracking methods where both targets were streaks were investigated; however, issues arose as the only difference is the skewed satellite travel vector compared to the star field. Since the model trained in this paper is looking for specific shapes in the images, accurate detections could not be made in this mode. As a result, only the two modes discussed previously were pursued. An example of this motion can be seen earlier among the labeled images in Figure 1.

While EfficientDetLite0 benefited from the lowest inference times, the confidence levels during object detection suffered a loss in accuracy in many cases. This concept held constant when looking at EfficientDetLite4 where there is a generally higher detection confidence at the cost of significantly longer inference times. The relationship between these two models was consistent across the the majority of the inference images, with only a few outlying cases where the models were close or reversed in detection performance. In situations where RSOs were near stars or were relatively faint in comparison to the other objects in the image, the EfficientDetLite models had the most difficulty. This is clear in Figures 4 & 5 where one of the RSOs is completely missed due to the proximity to the streaking star. The difficulty was also reflected in the MobileNetV2 results as, although detected, the confidence was at or barely above the imposed 20% threshold during testing. Similar challenges were present when the reference was switched as well since the brightness or single stars or star clusters can be difficult to distinguish from RSOs.

The result of this work is a CNN model that is be able to correctly detect satellites in star imagery (both simulated and real), while trained on a limited dataset. The performance of the model was checked by running inference on an external database of real satellite imagery and comparing the computation time and detection confidence. The detected locations of satellites in imagery can then be used as measurements for orbit determination. Having an effective model based synthetic imagery presents a number of benefits, primarily in the ease training data acquisition. Observation time and resources which would have otherwise been used for gathering training images could be used to enhance or extend final model deployment instead. Additionally, this method of satellite detection is ideal for embedded systems and mobile implementation where computational resources needed for inference can be limited. Note that the results in Table 2 do not take confidence values of each observation above the threshold into account for model comparison.

Based on this work, it has been shown that synthetic imagery can be used in practical object detection applications for tracking satellites. Due to the robust performance of the model on a small device, a more advanced implementation of this concept could be pursued where RSOs are tracked in real-time from mobile ground-based telescope observations. Future research can be done to optimize the detection of RSOs in non-ideal conditions where the objects are relatively dim or close in proximity to stars in the images. Further augmentations could also provide accurate automated orbit determination informed by satellite detection models.