Mitigating sampling bias in risk-based active learning via an EM algorithm

The approach to semi-supervised learning considered in the current paper aims to exploit a generative model form of statistical classifier, specifically a Gaussian mixture model, the details of which can be found in [1, 9]. Generative models can conveniently account for labelled and unlabelled data. This capability is achieved by modifying the expectation-maximisation (EM) algorithm [5], typically used for unsupervised density estimation, such that the log-likelihood of the model is maximised over both unlabelled and labelled data. For details of the approach, the reader is referred to [2].

Within the risk-based active learning algorithm, it is possible to apply EM every time a new unlabelled observation is acquired; however, this would be computationally expensive. To limit the computational cost of the modified active learning algorithm, the EM update was only applied following the retraining of the model subsequent to the acquisition of a new ground-truth label obtained via inspection.

In order to employ risk-based active learning for the development of a statistical classifier, a decision process for the structure $S$ must first be specified. For consistency with [9], an identical binary maintenance decision process is selected for the current study.

Consider an agent that, at discrete-time instances $t\in\mathbb{N}$, is tasked with making a binary decision $d_{t}$. The agent must select an action from $\text{dom}(d_{t})=\{0\text{ (do nothing)},1\text{ (repair)}\}$, such that some degree of operational capacity is maintained for $S$ at instance $t+1$. It is assumed that the agent has access to the observed discriminative features $\mathbf{x}_{t}$, and must infer the current and future latent health structural states $y_{t}$ and $y_{t+1}$ via a statistical classifier and a transition model, respectively. An influence diagram for such a decision process is shown in Figure 3.

As aforementioned, the agent requires a transition model in order to obtain a forecast for the future health state $y_{t+1}$. For the current case study, it is assumed that this model is known a priori so that focus can be on the development of the statistical classifier. Examples of this transition model being developed from data and prior knowledge of physics are presented in [10] and [8], respectively. As shown in Figure 3, the future health state $y_{t+1}$ is conditionally dependent on the current health state $y_{t}$ and the decision $d_{t}$. Given $d_{t}=0$, it is assumed that the structure will monotonically degrade with a propensity to remain in its current health state. This assumption is reflected in the conditional probability distribution $P(y_{t+1}|y_{t},d_{t}=0)$ presented in Table 2. Given $d_{t}=1$, it is assumed that the structure is returned to its undamaged state with probability 0.99 and remains in its current state with probability 0.01. This assumption is reflected in the conditional probability distribution $P(y_{t+1}|y_{t},d_{t}=1)$ presented in Table 2.

For simplicity, and for more compact influence diagrams, within the current case study it is assumed that health states may be mapped directly to utilities. Once again, the utility function used here is specified to reflect the relative utility values that may be expected in a typical SHM application. The utility function $U(y_{t+1})$ is provided in Table 4. It can be seen from Table 4, that for health-states 1 and 2, in which the structure is at full operational capacity, positive utility has been assigned. For health-state 3, which corresponds to a reduced operational capacity, a lesser positive utility has been assigned. For health-state 4, which corresponds to critical damage, a relatively-large negative utility has been assigned, to reflect a complete loss of operational capacity and an additional severe consequence associated with structural failure, e.g. environmental damage.

Figure 3 indicates that the actions in the domain of $d_{t}$ have associated costs, specified by the utility function $U(d_{t})$ and shown in Table 4. It is assumed that $d_{t}=0$ has no utility associated with it. On the other hand, $d_{t}=1$ has negative utility because of the expenditure necessary to cover the material and labour costs associated with structural maintenance. It is worth noting that, in many practical applications, the specification of utility function is non-trivial and is an active topic of research outside the scope of the current paper. Hence, for this case study, relative utility values are selected to be only somewhat representative of the SHM context.

As alluded to previously, a probability distribution over the current health state $y_{t}$ is inferred from the observed features $\mathbf{x}_{t}$ via a statistical classifier subject to risk-based active learning – for the current case study a four-class Gaussian mixture model is used. To fully specify the contextual parameters for risk-based active learning, it is assumed that the ground-truth health state at discrete-time instance $t$ can be obtained via inspection at the cost of $C_{\text{ins}}=7$. Here, cost can be interpreted as a strictly non-positive utility.

In order to assess the effects of sampling bias on decision-making performance and for comparison to highlight the effects of introducing semi-supervised learning, risk-based active learning was used to learn a GMM for use within the decision process outlined in the previous section. This process was repeated 1000 times. For each repetition, the dataset was randomly halved into a test set and a training set $\mathcal{D}$. From the training set, a small ($\sim$0.2%) random subset retain their corresponding ground-truth labels. These data form the initialised labelled dataset $\mathcal{D}_{l}$. The remaining majority of data from $\mathcal{D}$ have their ground-truth labels hidden, forming the unlabelled dataset $\mathcal{D}_{u}$.

Figure 4(a) shows a GMM from one of the 1000 repetitions after the risk-based active-learning process. From Figure 4(a), one can see that during the active-learning process, querying is concentrated in highly-localised regions of the feature space. Specifically, regions between the clusters for Class 3 (moderate damage) and Class 4 (severe damage) have been queried preferentially. It is clear from Figure 4(a) that the subset of data $\mathcal{D}_{l}$ is not representative of the underlying distribution indicating that sampling bias is present. Nonetheless, the queried data have been somewhat successful in learning a decision boundary - as can be deduced by observing the region of high EVPI between the means of clusters for Class 3 and Class 4 in Figure 4(b).

The EM approach to semi-supervised learning was incorporated into the risk-based active learning process and applied to the case study. Once again, 1000 repetitions were conducted, each with randomly selected training and test datasets and with $\mathcal{D}_{l}$ randomly initialised as a small subset of the training data.

Figure 5 shows a GMM for one of the 1000 runs after the risk-based active learning process incorporating EM.

It can be seen from Figure 5(a) that, similar to the GMM without semi-supervised learning, risk-based active learning with semi-supervised learning results in labels being obtained for localised regions of the feature space, with Class 3 (moderate damage) being preferentially queried. Nonetheless, this figure shows that the clusters learned in a semi-supervised manner fit the data very well. Furthermore, examination of Figure 5(b) reveals that the resulting EVPI distributions over the feature spaces distinctly differ from that in Figure 4(b). For the EM approach, the introduction of semi-supervised learning into the risk-based active-learning process has enabled the inference of a well-defined decision-boundary indicated by the band of high EVPI separating the $2\sigma$ clusters for Class 3 and Class 4.

Figure 6 shows how the number of queries varies between each approach to risk-based active learning. It is immediately obvious from Figure 6, that incorporating semi-supervised learning reduces the number of queries made substantially. The significance of this result becomes most apparent if one recalls that, in the posed SHM decision context, the number of queries can be mapped directly onto inspection expenditure. This result is to be expected as, when employing semi-supervised learning, each time a query is made, additional information from the unlabelled dataset is utilised to define a class. This allows clusters to become well-defined more quickly, reducing the area of high-EVPI regions (as is visible in Figure 5), meaning fewer queries are made later in the dataset. This result is evident from Figure 7.

Figure 7 compares the total number of queries for each index in $\mathcal{D}_{u}$ over the 1000 repetitions of risk-based active learning conducted with a GMM, and a GMM with EM. It can be seen from Figure 7, that the incorporation of semi-supervised learning into the risk-based active approach results in relatively more queries being obtained during the first occurrence of each class, with relatively fewer queries being made at later occurrences. It can be seen that Class 3 is heavily investigated at each occurrence when semi-supervised methods are not employed. This phenomenon is to be expected when one considers the differences between the high-EVPI regions shown in Figure 4(b) and Figure 5(b); as previously discussed, semi-supervised learning results in a well-defined decision boundary, thereby reducing the likelihood that data will have high value of information.

Figure 8 provides a comparison of the median decision accuracies and $f_{1}$-scores throughout the querying process between risk-based active learning, with and without semi-supervision.

Here, the decision-making performance achieved via each classifier is assessed via a quantity termed ‘decision accuracy’. The decision accuracy is a comparison between the actions selected by an agent using the classifier being evaluated, and the optimal (correct) actions selected by an agent in possession of perfect information. For details of this performance measure, the reader is directed to [7].

From Figure 8(a), one may be inclined to deduce that the introduction of semi-supervised learning has been detrimental to the performance of risk-based active learning as the introduction of EM appears to delay the observed increase in decision accuracy. However, when considered alongside Figure 7, one can realise that, because semi-supervised learning results in increased querying early in the dataset, decision accuracy over the whole dataset is improved. Furthermore, a decline in decision performance is not observed for the algorithm incorporating semi-supervised learning; this result is because of the reduction in sampling bias obtained via the inclusion of unlabelled data. Figure 8(b) shows the $f_{1}$-score classification performance, these results provide further indication that the models learned via semi-supervised risk-based active learning better represent the underlying distribution of data and that the detrimental effects of sampling bias have been reduced.

In summary, the case study has shown that, for some applications, semi-supervised learning provides a suitable approach to reducing the effects of sampling bias. The generative distributions obtained via these semi-supervised risk-based active-learning approaches better fit the underlying data distributions, whilst also establishing well-defined decision boundaries in a cost-effective manner.