Online Active Learning for Soft Sensor Development using Semi-Supervised Autoencoders

In industrial operations, soft sensors are frequently used for real-time prediction of hard-to-measure process variables, as well as to support system backup strategies, what-if analysis, sensor validation, and fault diagnosis (Fortuna et al., 2007). Soft sensors are classified into two types: model-driven sensors, which are used to depict the ideal steady-state of a process under normal operating conditions, and data-driven sensors, which are used to better approximate real process conditions (Kadlec et al., 2009). Many labeled observations are required for training the regression models used in soft sensor development, but in industrial contexts, data is often abundant only in an unlabeled form. Obtaining product information can be both expensive and time consuming, as it may necessitate the intervention of a human expert or the use of expensive testing equipment. As a result, active learning is becoming increasingly useful for reducing the number of labels required to achieve compelling predictive performance. Active learning-based sampling schemes use some evaluation criteria to assess the informativeness of the unlabeled data points and prioritize the labeling of the most useful instances for building the model. Three macro scenarios can be identified depending on how the unlabeled instances are fed into the learner and then selected to be labeled by an oracle (Settles, 2009). The first scenario is referred to as membership query synthesis, and it allows the learner to query the labels of synthetically generated instances rather than those sampled from the process distribution. The second scenario is stream-based active learning, also known as selective sampling. It denotes a situation in which instances are drawn sequentially and the learner must immediately decide whether to keep the instance and query its label or discard it. The third and final scenario is pool-based active learning, which depicts a situation where a large amount of unlabeled data is collected all at once and made available to the learner, which can rank all of the data points and select the most informative ones. While many researchers have been working on active learning in the latest years, pool-based active learning for classification has received the most attention (Cai et al., 2013).

In this work, we focus on stream-based active learning (Cacciarelli and Kulahci, 2023), which represents a more difficult task as the learner cannot observe all of the available observations before deciding which labels to query. We believe that this scenario accurately reflects high-volume production processes in which samples are processed very rapidly and labels are no longer retrievable. Stream-based active learning should be considered and prioritized for all industrial processes with similar properties.

In regression modeling, we try to learn a function $\hat{f}:\boldsymbol{x}\in\mathbb{R}^{p}\rightarrow y\in\mathbb{R}$ to predict a quality characteristic or a hard-to-measure variable $y\in\mathbb{R}$ that is related to other process variables $\boldsymbol{x}\in\mathbb{R}^{p}$. Accordingly with many active learning approaches (Cai et al., 2013), we assume a labeled dataset $\mathcal{L}=\{(\boldsymbol{x}_{i},y_{i})_{i=1}^{n},\boldsymbol{x}_{i}\in\mathbb{R}^{p},y\in\mathbb{R}\}$ with $n$ observations is initially available to fit a linear regression model of the kind

where $x_{0}=1$ is the intercept term and $x_{i}$ with $i=1,...,p$ are the $p$ process variables. Parameters are estimated by minimizing a squared error loss given by

After an initial model has been built, we aim to acquire additional observations by evaluating the unlabeled data points, until a budget constraint is met. Some commonly encountered approaches are presented below.

Given the impossibility of ranking unlabeled instances in real-time and deterministically optimizing the sampling criteria, we propose leveraging unlabeled data to impose a threshold, or upper control limit (UCL), on the informativeness of the incoming data points. The unlabeled data pool can be acquired by either observing the process for a period of time without sampling the product information $y$ or by using data that is already available in the form of a historical database $\mathcal{H}=\{(\boldsymbol{x}_{i}),\boldsymbol{x}_{i}\in\mathbb{R}^{p}\}$. The primary difference between pool-based active learning and online active learning is that the labels of observations pertaining to $\mathcal{H}$ can no longer be queried because they only exist digitally, and the associated physical part or component is no longer available. The data in $\mathcal{H}$ is used to estimate the distribution of the statistics employed by the criteria in Equations 2, 3, and 4. In this study, we employed kernel density estimation with a Gaussian kernel. The UCL is then determined by specifying the appropriate sampling rate $\alpha$. For a given criterion $\mathcal{J}$, the threshold is defined as

Using the UCL obtained from Equation 5, we should then only collect the $\alpha$-percent most informative data points, according to the specific criterion $\mathcal{J}$ (Cacciarelli et al., 2022, 2023). In this work, we test the stream-based active learning routine using Mahalanobis distance, ambiguity, and expected model change as sampling criteria $\mathcal{J}$. Before starting the active learning routine and collecting additional observations, we also propose to use a semi-supervised architecture by training an autoencoder network on the historical data $\mathcal{H}$. With semi-supervised learning, we can exploit all the available unlabeled data and learn how to extract relevant features that could be better predictors than the raw input features. Indeed, when variables are highly correlated, it has been demonstrated in the literature that a PCR model can be enhanced with semi-supervision (Frumosu and Kulahci, 2018). With regards to deep learning methods, autoencoders have been proposed to deal with semi-supervised learning in fault classification (Jia et al., 2020; Jiang et al., 2017). Recently, autoencoders have also been investigated in soft sensors applications (Yuan et al., 2018; de Almeida Moreira et al., 2021) but their contribution to the stream-based active learning scenario has not been evaluated yet. In this work, we propose the use of a semi-supervised architecture as the one shown in Figure 1. An orthogonal autoencoder (OAE) is employed for feature extraction and the encoded features are then used as predictors in a linear regression model. An OAE is an autoencoder network that minimizes an Ortho-Loss, which is comprised of a squared reconstruction loss and an orthogonality regularization term (Wang et al., 2019; Cacciarelli and Kulahci, 2022). The regularization, weighted by a parameter $\lambda$, encourages the network to learn uncorrelated features in its bottleneck. This is particularly beneficial to alleviate the multicollinearity issue in the regression modeling stage.

The main advantage of the semi-supervised model is that the extracted features are more expressive than the original process variables. However, if the dimensionality of the bottleneck is lower than the one of the input features, there is an additional benefit for active learning. Indeed, because the majority of the provided active learning approaches are model-based, an initial number of labels is required. QBC and EMC, in particular, employ the linear regression model’s predictions to select the data points that should be queried. To uniquely determine the coefficients of a regression model, we need a number of observations larger than the number of parameters $\boldsymbol{\beta}$ to be estimated. This initial set of observations is usually collected at random (Cai et al., 2013). As a result, by reducing the dimensionality of the parameters $\boldsymbol{\beta}$, we will be able to anticipate the active collection phase and get more robust estimates for the same experimental cost. The complete active learning routine is reported Appendix A.

The Tennessee Eastman Process (TEP) is considered the gold-standard benchmark for testing process control approaches (Ricker, 1996; Capaci et al., 2019) and, recently, it has also been used for validating active learning and soft sensor development methods (Zhu et al., 2015; Grbić et al., 2013). The variables that have been used as predictors in regression modeling are the same 16 controlled process variables used by Zhu et al. (2015) and Grbić et al. (2013), and the continuous response is Stream 9E, a composition measurement belonging to the purge stream.

Figure 2a shows how the suggested semi-supervised architecture can increase the predictive performance. Data is randomly sampled in both situations, but the two linear models are fitted using the original process variables and the features extracted by the OAE. To ensure comparability between the two learning curves, we fitted the first model when the number of gathered observations exceeded 16. We believe the improvement is due to the OAE’s ability to express nonlinear relationships in data in its encoded features and to the fact that with the extracted features, we have the same number of observations to estimate a smaller number of parameters. In Figure 2b, we try to improve the semi-supervised result by using the proposed active learning strategies. It is clear that EMC and QBC consistently outperform the passive random approach in terms of recommending the most informative data points. On the contrary, the Mahalanobis distance appears to worsen the predictive performance. We believe this may due to the fact that data points with high $T^{2}$ statistics may be outliers, whose inclusion in the training set eventually degrades performance. It should be noted that in Figure 2b all the sampling strategies use the features extracted by the OAE. Experimental setup and training details are reported in Appendix B.

Industrial data is often only available unlabeled as quality inspections and manual annotation tasks are costly and time-consuming. In this work, we proposed a semi-supervised model based on OAEs for extracting relevant features and reducing multicollinearity. On top of this, we reviewed and adapted for the online setting some of the most widely used active learning strategies for linear regression. The analysis demonstrates how properly using the historical data and taking into account the expected response allows for a faster reduction of the prediction error. For future research, we will consider more advanced architectures such as LSTM autoencoders or transformers to obtain encoded features that take into account the temporal dependency in the data.

The data is generated using the MATLAB code provided by Reinartz et al. (2021) and Andersen et al. (2022) with the Ricker closed-loop simulation model. No faults have been introduced throughout the 50 simulation runs, which are generated providing different seeds to the simulator. The variables used are reported in table 1. Sample rate was set to 1 minute.

The active learning routine was tested once on each of the 50 simulation runs. Figure 2 reports the mean and standard deviation for each method across these 50 runs (shaded regions indicate $\pm$ 1 standard deviation).

With regards to the autoencoder structure, we used an encoder whose dimensionality of the layers corresponds to [16, 160, 80, 40, 20, 10]. The decoder is symmetrical to the encoder. The penalty term corresponding to the weight of the orthogonality regularization in the loss function was set to 0.10. No fixed number of epochs was used for the training as we followed an early stopping approach, setting a patience of 10 on the number of accepted epochs without improvement on the validation loss (20% of the training data is used for validation). Finally, the bandwidth used for the kernel density estimation of the UCL is found using Scott’s rule (Scott, 1992).