Adversarial Attacks on Machinery Fault Diagnosis

There are many fault diagnosis datasets. The Bearing dataset, from Case Western Reserve University (CWRU) bearing data center, is completed by Case Western Reserve University. As the most widely used standard dataset for bearing vibration signal processing and fault diagnosis, the fault features of CWRU Bearing Datasets are obvious and the related references are abundant. In this work, we used the Drive End (DE) part of the CWRU dataset, with a 12KHz sampling rate, whose 10 categories include 9 faulty types and 1 normal type. The 9 faulty types, as shown in Table 1, indicate the fault diameter, from 0.007 to 0.021 inches, and the faulty location of a bearing, inner race, ball, or outer race.

Zhang [15] used a convolutional neural network named WDCNN shown in Figure 2, in which the first layer convolutional kernels were large, and the rest are small, to diagnose faults on CWRU Bearing dataset. This pattern allows the model to focus on the global features and reduces training time by avoiding a large number of convolutional layers. Additionally, batch normalization (BN) makes the model easy to train.

Turker Ince [5] applied an improved CNN to the raw data (signal), eliminating the need for a separate feature extraction algorithm. This method is efficient in terms of speed and hardware. The experimental results demonstrated its effectiveness in monitoring motor conditions in real time, compared with traditional machine learning methods.

M. Zhao [16] developed a variant of deep residual networks (DRNs), the so-called deep residual networks with dynamically weighted wavelet coefficients to improve diagnostic performance, which takes a series of sets of wavelet packet coefficients on various frequency bands as an input.

Some other models, also work well on CWRU Bearing dataset. For example, LeNet, AlexNet, BiLSTM can also achieve the accuracy of more than 99% [17]. All the above models mentioned acted as victim models in our experiment to evaluate the robustness of fault diagnosis.

Existing methods can be categorized as, gradient sign-based, optimization-based, evolutionary-based, or generate adversarial networks (GAN)-based [18, 19, 20]. We used gradient sign-based methods to generate adversarial examples. Also, with different assumptions, background information and restrictions, we can divide the methods based on the following three aspects.

White-Box and Black-Box. For white-box, information on models, including the dataset, network architecture, weights and hyperparameters of the models, is accessible. While in most scenarios, it’s impractical. And for black box case, only the output of the model can be exploited. In this paper, we assume that the information of the models is accessible.

Untargeted and Targeted. Untargeted attacks aim to make the attacked models misclassify the input. And targeted attacks have a specific requirement for the output, which means that with additional perturbations, the models must transcribe the input to the targeted class. In fault-diagnosis conditions, the aim is to change the condition of the output from normal to faulty or vice versa. In this paper, both patterns were considered.

Universal and Individual. A perturbation generated for all samples functions universally for the whole dataset. Most of the existing attacks focus on individual attack based on a specific input. In this paper, the individual perturbation is used to fool the fault diagnosing classifiers.

Fast Gradient Sign Method (FGSM). Goodfellow et al. proposed the approach, which can generate untargeted adversarial examples [21]. Given the vibration signals $x$ of the fault-diagnosis system and its corresponding faulty type $y$, perturbation $\delta$ can be expressed as

where $\theta$ denotes the parameters of fault-diagnosis system, and $J(\cdot)$ is the loss function. Given the target faulty type $y^{{}^{\prime}}$, this method can also generate targeted adversarial examples by adding a perturbation

Projected Gradient Descent (PGD). FGSM iterates for once, while PGD adopts many small iterations

where $clip(\cdot)$ means that the perturbation must be restrained within the required scope from $1$ to $0$. Also, PGD can similarly obtain targeted faulty type [22].

Distortion Measure. For image and audio, whose information can be directly understood by humans, the distortion measures include the signal-to-noise ratio (SNR) and $L_{2}$ distance can guarantee the perturbations as imperceptible as possible. Since vibration signals are intricate, traditional distortion measures for adversarial examples are not applicable. The way to add noise restricts the operation of the attacks in fault-diagnosis condition is how to add noises. In reality, noises may emerge in the following forms: (i) The mechanical movement of electrical appliances. (ii) The circuit of the equipment within the system. (iii) The malicious attack from the computer virus to change the original data. (iv) The physical attack by imposing an external force on the sensors. Therefore, the attack cost can be understood as the external energy exerted on the vibration source and thus the energy $E(s)$ of the signals can be expressed as

where $x(k)$ denotes the sampling points of the origin signals. We define the measure of the attack cost as

where $x^{{}^{\prime}}$ is the generated examples, $n(k)$ is the sequence of the noise, $S$ is the size of the segment, and $s$ is the start location of each segment. This measure denotes the degree of the attacks with maliciously designed or just environmental perturbations.

Data Process. The dataset was divided into training, validating and testing part with the ratio of $0.6,0.2,0.2$. In our experiment, each class has 1000 examples and each example has $2048$ normalized sampling points to make the distributions of all data similar and models converge quickly. In this paper, we improved the data processing used by Zhang to constrain the values of the signals between 0 and 1 [15]. The data after processing is shown in Figure 3, where there are some distinguishable differences between some classes, for example, class Normal and class $OR007$. Therefore, it is reasonable that the attack success rate from the faulty to normal is low, while the conversion between different types of faulty signals can be easily accomplished with a high success rate.

Pretrained Victim Models. First, we pretrained the victim models to get the basic accuracy. After 1000 epochs’ training, all of the models can achieve the accuracy of more than $99\%$ on testing dataset, where the worst is $99.43\%$. Therefore, it’s obvious that the models can fit the dataset well.

Untargeted Attack. We generate untargeted adversarial examples with the test dataset to misclassify the models. The success rates of FGSM, and PGD are shown in Table 2. As we can see in Table [tab:tab2], models such as WDCNN can be attacked with high accuracy and restricted cost. Hence, it is justifiable to think that DNNs-based fault diagnosis models are vulnerable to adversarial attacks. Additionally, from the confusion matrix given in Figure 4. Instead of having random distribution, the misclassified classes were partial to certain classes such as $B014,B021,IR021$ and $OR021$. The partial predicted classes are marked with dark blue, which shows that the conversion difficulty between different types varies largely.

Targeted Attack. We generated a perturbation to fool the models with targeted attack methods, with the results shown in Table 3. The attack success rate varied greatly across models, but the success rate of one model, AlexNet, was extraordinarily higher than others. Based on this discovery, we proposed a simple scheme to defend against the malicious attacks.

Depending on the above results, it is evident to conclude that the robustness varies with the differences of the models, when targeted attacks are conducted. However, we go a step further to discover what plays a major role in this difference. By comparing AlexNet with other models such as WDCNN, we found that the proper use of data normalization can facilitate convergence, and defend against the potential attacks, because the robust models shown in Table 3 have all used data normalization. Based on the above conclusion, we combine their processing procedures and propose a defense method, as shown in Algorithm 1. The steps shown in Algorithm 1 can universally be used as a defensive layer for NN-based fault-diagnosis models.

To validate the defensive ability of the proposed method, we constructed this data process as a defensive layer, and added a defense layer to AlexNet before the first convolutional layer, to defend against the attacks of FGSM and PGD. The results are shown in Figure 5: AlexNet with this defense strategy has more robustness, regardless of the attack methods. This demonstrates that our method can effectively defend against intentional attacks. Moreover, to avoid the influences of the model structure and hyperparameters, we remove the data normalization of other models to contrast them with the original models. We found that the success rates had varying degrees of increasements after being remove the data normalization.

From the result of the proposed scheme, it is obvious that DN works well. In fact, the similar phenomenon was also observed recently in the image domain [23], but the defensive effect of similar method increases greatly in machinery vibration signals domain. To explain this result, we found that DN can map inputs to other fields, amplifying small sample differences due to data processing. Therefore, the amplified differences can act as crucial features to distinguish classes. The models without defensive layers, however, lost their robustness for the differences between the adversarial and the original inputs are small. Additionally, DN can also enhance the robustness of models by mitigating the instability during adversarial training [24]. Future works can be done to extend the application of DN in other fields.