Real-time Safety Assessment of Dynamic Systems in Non-stationary Environments: A Review of Methods and Techniques

Mon-stationary environments refer to situations where the underlying dynamics governing the system change over time, which may due to various reasons, such as changes in the system parameters, external disturbances, or sudden events. In the context of industrial applications, non-stationary environments are common in manufacturing processes, where the system dynamics can change due to variations in the raw materials, machine wear and tear, and changes in the production line setup. For example, in the semiconductor manufacturing process, variations in the process parameters can lead to changes in the device characteristics, affecting the overall quality of the product. Similarly, in transportation applications, non-stationary environments can arise due to changes in the traffic patterns, road conditions, and weather conditions. For instance, sudden changes in weather conditions such as heavy rainfall or snow can affect the road conditions and the driving behavior, leading to an increased risk of accidents.

One of the key characteristics of RTSA is its real-time nature, which requires the monitoring and analysis of system behavior to be performed in a timely and efficient manner. This poses significant challenges, especially in non-stationary environments, where system behavior can change rapidly and unpredictably. As a result, RTSA methods need to be adaptive and able to handle non-stationary data streams. Generally, RTSA involves the continuous monitoring and analysis of system behavior to detect potential safety hazards in real-time. The real-time safety level of a dynamic system should be predicted based on the monitoring data and environment factors, which can be typically defined as the impact degree of its operation on potential damage to surrounding people and the environment. In this case, potential analysis of risks plays an important role in related technologies. It is important to note that RTSA is not limited to any specific type of dynamic system and can be applied to a wide range of domains, including but not limited to industrial processes and transportation systems.

Clearly, it is crucial for the effectiveness of RTSA tasks based on the analysis of data stream. Considering the the processing capacity, specific requirements, and characteristics of the system being monitored, two types of RTSA tasks can be divided, which are:

1. $\bullet$

   Chunk-level RTSA (CRTSA): CRTSA methods process the data streams in batches, which allows for more efficient processing and the use of statistical techniques that require a larger sample size [21]. It is especially important when dealing with high-frequency data streams, where processing each individual sample would be computationally intensive.

2. $\bullet$

   Instance-level RTSA (IRTSA): IRTSA process each individual sample separately, which allows for a more detailed analysis of each data point. It is beneficial when the system behavior is highly dynamic and requires immediate detection of safety hazards. Instance-level methods may also be more suitable for low-frequency data streams, where the amount of data is smaller and processing individual samples is more feasible. Considering the granularity of information processing, IRTSA often brings higher labeling costs and computational complexity.

The main difference between these two approaches is the amount of information that is analyzed. Chunk-level RTSA tasks are typically used when the sampling frequency is very high and a large amount of data is being processed. In this case, statistical analysis and other indicators are used to analyze the data in chunks, which allows for a more efficient analysis. In contrast, instance-level RTSA tasks focus on analyzing each individual sample in the data stream. As illustrated in Figure 1, the primary logic flow of CRTSA and IRTSA differs. CRTSA primarily employs statistical information to aid in the query process, while IRTSA typically relies on prediction to assess the value of samples. While this approach involves more information than chunk-level analysis, it typically yields higher overall evaluation results.

In this context, it is important to note that the choice between these two types of RTSA tasks depends on the specific application and the nature of the data being analyzed. Careful consideration should be given to the trade-off between the amount of information analyzed and the accuracy of the evaluation results.

Although the considerable importance of RTSA in non-stationary environments, several prominent issues require careful consideration. These issues include, but are not limited to, the following:

Online active learning is an essential technique that facilitates real-time safety assessment by intelligently selecting and querying the most informative data samples. It reduces the need for manual annotation while maintaining the learning efficiency of the model [26]. With the aid of online active learning, real-time safety assessment models can adaptively incorporate new data, effectively improving their accuracy and responsiveness to changing environments [27, 28].

In IRTSA tasks within non-stationary environments, the online active learning approach enables the evaluation of individual data points as they are received in real-time [29]. Consequently, this assessment method may prompt annotation requests and model update processes during critical moments, such as when drift occurs. The model can then undergo incremental updates based on a select few valuable samples. Given the dynamics of the underlying distribution and the presence of noise interference, such a paradigm may, in some cases, yield superior results in comparison to supervised learning techniques [30].

Regarding chunk-level real-time safety assessment tasks in non-stationary environments, two strategies can be mainly considered. On the one hand, we can approximate the necessity of labeling incoming data chunks by assessing their value. Entire data chunks may be discarded when deemed unnecessary. On the other hand, we can allocate annotations to high-value samples at a fixed proportion within each incoming block, utilizing pool-based active learning approaches in such cases [31]. As a result, the primary challenge lies in evaluating the value of incoming data chunks and the samples they contain. When compared to IRTSA, such a paradigm can more effectively leverage statistical information to facilitate the design of value estimators.

In recent years, reinforcement learning has increasingly been applied to online active learning and may become a prominent development trend in the coming years. Balancing exploration and exploitation is particularly suitable for scenarios with concept drift. While blind exploitation can consistently yield satisfactory historical results, as concepts shift, those once satisfactory results may no longer be adequate. Exploration, however, effectively addresses this issue, continuously uncovering new information and potentially discovering improved solutions when concept drift occurs [32, 33, 34].

In the context of real-time safety assessment, online semi-supervised learning serves as an effective approach that leverages both labeled and unlabeled data. By efficiently utilizing the abundant supply of unlabeled data available, this method allows models to generalize better, enhancing their performance on safety-related tasks. Furthermore, online semi-supervised learning enables continuous model improvement, reducing the time and effort required for manual data labeling [35].

In general, semi-supervised learning (SSL) constitutes a significant class of machine learning techniques that seek to leverage unlabeled data for learning tasks. This approach has been extensively researched in batch learning settings, with detailed surveys available [36]. However, in the context of online learning, there are two main areas of research. The first is the transformation of traditional batch SSL methods into online algorithms capable of processing data streams comprising labeled and unlabeled data. We refer to this as online manifold regularization. The second area of research explores classical online learning tasks in transductive learning settings, where the assumption is that unlabeled data is available before online learning tasks commence. This area is called transductive online learning.

It is worth noting that online active learning is a specialized form of online semi-supervised learning, wherein an online learner handles both labeled and unlabeled data. Several studies integrate these two approaches into a unified framework.

Online transfer learning is another vital technology that bolsters real-time safety assessment capabilities. By leveraging pre-trained models and knowledge acquired from related tasks, online transfer learning accelerates the learning process and helps safety assessment models adapt to new environments quickly. This rapid adaptation is particularly crucial in safety-critical applications where timely and accurate decision-making is of paramount importance.

In order to enhance the performance of models in target domains, two prevalent methods are employed: sample weighting and feature matching [37]. Sample weighting involves reusing samples from the source domain, evaluating their values, assigning corresponding weights, and then training new models or updating existing ones accordingly. In recent research, Jin et al. trained weighted networks through source-target joint meta-learning, assigning higher weights to source regions that aid in target fine-tuning [38]. A double-weighted multi-source transfer learning approach , termed DRMTL, was then proposed based on a regularized least squares classifier [39]. Ahmadvand et al. applied a novel instance re-weighting scheme based on graph optimization to match the value distribution of source sample weights [40]. While in [41], a kernel mean matching method founded on individual correlation was introduced for sample weighting.

Feature matching, on the other hand, addresses the issue of differing feature spaces across domains by mapping the source domain to the target domain, or mapping both domains to a new one. The objective is to ensure the marginal and conditional distributions of source and target domains are as consistent as possible while preserving properties and identifying correlations between features. To measure feature similarity between source and target domains, metrics must be introduced. Commonly used indicators include Maximum Mean Discrepancy (MMD), Kullback-Leibler Divergence, and Jensen-Shannon Divergence [42, 43]. Given the exceptional feature extraction capabilities of neural networks, they are highly suited for this method [44, 45, 46].

Finally, online anomaly detection plays a pivotal role in real-time safety assessment by monitoring data streams and identifying irregularities or unexpected behaviors that may signify safety hazards. By providing early warnings and facilitating proactive interventions, this technique ensures safety by allowing for timely corrective actions to be taken in response to potential threats. In a non-stationary environment, the occurrence of a change results in an anomaly in comparison to the environment before the change. The use of online anomaly detection technology to detect changes in the environment and promptly adjust the model can enhance decision-making by providing more accurate recommendations [47].

Online anomaly detection technology can be broadly categorized into two types: sample-based and distribution-based methods. The sample-based methods involve identifying abnormal samples from a set of samples, whereas distribution-based methods detect changes in the distribution of a set of samples relative to another set of samples. In the former technique, it is common to measure the degree of deviation of a sample from the mean. For instance, the Otsu algorithm, which is used for image binarization, employed the maximum variance of black and white pixels to establish the threshold for identifying abnormal samples [48]. Hundman et al. utilized an index that maximizes the mean, standard deviation, and continuous sequence to determine the threshold and perform anomaly detection [49]. Siffer et al. used the extreme value theory to detect anomalies, fit the distribution of extreme values using available data, and identify new data as abnormal based on the distribution [50]. Additionally, the Gaussian distribution is often used in sample-based anomaly detection techniques [51, 52, 53]. Distribution-based techniques, on the other hand, usually utilize statistical methods to determine changes in the distribution of samples. Hypothesis testing is a popular statistical method used to assess whether two distributions are equivalent [54]. The relative entropy methods are commonly used to measure the difference between two distributions [55, 56].