Self-Supervised Learning for Effective Denoising of Flow Fields

This study used two DNS flow data sets with synthetic noise to evaluate the DL model. First, the 2D laminar flow around a square cylinder (bluff body flow) at $Re_{D}$ = 100 were used as a demonstration case. The turbulent channel flow at $Re_{\tau}$ = 180, 395, and 550 were used for subsequent denoising performance tests of MS-DAE.

In the bluff body flow, the simulation domain size was set to $x_{D}$ × $y_{D}$ = 20$D$ × 15$D$, where $x$ and $y$ are the streamwise and spanwise directions, respectively. The corresponding grid size was 381 × 221. The stretching mesh technique applied local mesh refinement near the cylinder walls. The uniform inlet velocity and pressure outlet boundary conditions were applied to the inlet and outlet of the domain. Furthermore, no-slip boundary conditions were used in the cylinder walls and the symmetry plane to the sides of the domain. The dimensionless time step of the simulation, $u_{\infty}\Delta t/D$, was set to $10^{2}$. The DNS data used in this study were cut around the edges and interpolated into uniform grids, where the new domain size is $x_{D}$ × $y_{D}$ = 17$D$ × 8$D$ and the uniform grid size is 256 × 128. Moreover, the interval between the collected snapshots of the flow fields i s ten times the duration of the simulation time step. A total of 1200 snapshots were collected and used.

For the turbulent channel flow, three friction Reynolds numbers, namely $Re_{\tau}$ = $u_{\tau}\delta/\nu$ = 180, 395, and 550, were used. Here, $u_{\tau}$ is the friction velocity, and $\delta$ is half of the channel height. Table 11 lists the simulation parameters of each $Re_{\tau}$. The periodic boundary condition was used in the streamwise and spanwise directions. Moreover, the no-slip condition was applied to the upper and lower walls of the channel. The turbulence statistics obtained from the DNS were validated by comparing them with the results from reference paper[2, 3].

Training and testing data were collected from the $y$–$z$ plane of the 3D channel flow domain. The flow data at $Re_{\tau}$ = 395 and 550 were interpolated to align with the grid size of the DNS data for the flow at $Re_{\tau}$ = 180, which features 128 and 256 grids in the $y$- and $z$-directions, respectively. This data processing ensures that the flow at all three Reynolds numbers can be input into the DL model simultaneously.

In this study, synthetic noises, namely Gaussian noise, salt-and-pepper noise, and speckle noise, were added to degrade the quality of all DNS data cases. For Gaussian noise, zero-mean random noise with $\mathcal{N}$(0,$\sigma^{2}$) was added to the flow fields, as displayed in Figure 1 (a), where $\mathcal{N}$, $\sigma^{2}$ are the normal distribution and variance, respectively. Salt-and-pepper noise data are generated by randomly adding the minimum and maximum values of the velocity components to the flow data, as displayed in Figure 1 (b). Speckle noise is a granular texture degrading quality due to interference among wavefronts in coherent imaging systems, such as optical coherence tomography [38]. This study simplified speckle noise by assuming it exists only at the particle positions in the PIV system. Thus, artificial speckle noise is generated using a method displayed in Figure 1 (c). First, an artificial particle background, where the density of the particles was 50%, imitating the real PIV measurement, was devised. Next, all pixels with particles (white) were considered as coefficient 1. By contrast, the black region without particles was coefficient 0. Finally, speckle noise data were obtained by multiplying the background coefficient with Gaussian noise and adding it to the flow data. Moreover, the SNR represents the noise level. Here, SNR=$\sigma_{DNS}^{2}/\sigma_{noise}^{2}$, where $\sigma_{DNS}^{2}$ and $\sigma_{noise}^{2}$ are the variances of the DNS and noise data, respectively. This study defines the noise level as the reciprocal of the SNR, 1/SNR.

Noisy PIV data of flow over a square cylinder were generated and used in this study. Noisy PIV data were generated through a return-type water channel. The test section size of the water channel was 1 m (length) × 0.35 m (height) × 0.3 m (width). The free stream velocity was set to 0.02 m s-1 with an $Re_{D}$ of 200. Background noise was generated at high levels because of external noise and the sparse honeycomb configuration of the water channel. Polyamide12 seed particles of INTECH SYSTEMS were used to seed the channel with a 50 $\mu$m diameter. A high-speed camera (FASTCAM Mini UX 50) and a continuous laser with a 532 nm wavelength were used in the PIV system. The snapshot frequency was set to 24 Hz. In this experiment, a square cylinder model was constructed using an acrylic board, and the cross-section of the model was set to 1 × 1 cm. The model was not entirely transparent. Thus, a shadow region existed near the bluff body when the laser passed through the model.

Autoencoder (AE) is a classical neural network extensively used for image classification, object detection, and natural language processing [30]. AE solves various problems in fluid dynamics, such as the super-resolution reconstruction of flow fields [39] and predicting turbulent dynamics [40]. Furthermore, in this study, AE’s fluid flow denoising ability was introduced. The denoising mechanism of the autoencoder was investigated by comparing various networks with different structures. This comparison used 1000 and 200 snapshots of noisy bluff body flow data with Gaussian noise at 1/SNR = 0.5 as training and validation data, respectively. The training of the AE was based on self-supervised learning without clean label data, in which the input and output were the same. Thus, the testing data were the same as the training data. Table 2 summarises the comparison, which indicates that only bottleneck-shaped AE with convolutional neural networks (CNNs) and a small latent space size can denoise the data. Such AE is called DAE.

Figure 2 displays the structure of the DAE. The convolution and carpooling layers make up the encoder part of DAE. The decoder is composed of convolution and sampling layers. The channel number of the convolution layers varies with the AE depth, where the CH is larger when the convolution layers are closer to the latent space. Furthermore, noisy and clean data were passed several times into the DAE to output procedure data from various convolution layers. As displayed in Figure 3, the noise level was reduced with the output from various convolution layers in the encoder. This phenomenon proved that the DAE encoder can filter the noise as a funnel. Therefore, only bottleneck-shaped AE can denoise rather than bottlebody-shaped AE. When noisy data passed the encoder, the data size was compressed and decreased by pooling layers. The filters of the convolution layer were not sufficiently sensitive for capturing high-frequency noise features. Thus, noise information cannot be preserved in the encoder. If noisy data were not compressed and filtered sufficiently with a large latent space, noise information was retained so that the AE loses its denoising ability. The noise of the output around the latent space increased slightly because the procedure data around the latent space were highly compressed and sensitive to 1/SNR calculation. The procedure data in the decoder exhibited a low noise level. Finally, the denoised data were reconstructed.

The DAE in the last subsection has a simple frame that can only denoise uncomplex flow fields. The performance of this DAE for denoising the turbulent flow was not satisfactory, as the DL model could not distinguish small eddies from noise and could not reconstruct the turbulent flow well. Thus, a complex deep multiscale DAE (DMDAE) was developed. As shown in Figure 4, DMDAE features deep convolutional layers, enhancing its capability to capture features. The skip connexion was used to avoid the vanishing gradient problem in the training process. Furthermore, the encoder and decoder have three branches with various filter sizes. This multiscale CNN strategy renders the convolutional layers distinguishable [41].

This subsection introduces the training and testing processes of DAE and DMDAE. Before training, all data were normalized using the min–max normalization function using the minimum and maximum values from the average flow field to scale the values between 0 and 1. Considering DAE as an example, noisy data were passed into DAE to obtain output data. Next, the training loss was calculated from the output and noisy input data using the mean square error loss function. After calculating the training loss, the optimization algorithm updated the weights of the DAE to minimize the loss throughout the training period. This study used the adaptive moment estimation (Adam) algorithm as the optimization algorithm [42]. Validation data were used to validate the DL model by calculating the validation loss. The weights were saved only if the validation loss decreased compared with the previous epoch. On completion of the training process, testing data (identical to the training data) were inputted into the trained DL model to obtain denoised flow data. This study used the open-source library TensorFlow 2.3.0 to implement the DL model. The customized sample Python code for the proposed models can be accessed on the website (https://fluids.pusan.ac.kr/fluids/65416/subview.do).

In this subsection, noisy bluff body flow data with Gaussian noises at 1/SNR = 0.1, 0.5, and 1.5 were used to train the DAE, where each noisy data set includes 1200 snapshots. In the testing process, noisy bluff body flow data at various noise levels from the training data were used to check the denoising performance and DAE generalization. In addition to the test using training data, noisy data at 1/SNR = 0.3 and 1 were used for the interpolation test, whose noise levels were within the noise level range of the training data. Noisy data at 1/SNR = 0.05, 0.075, 1.75, and 2 were used for the extrapolation test, whose noise levels were out of the noise level range of the training data.

Figure 6 displays the denoised instantaneous contours of bluff body flow with Gaussian noise at 1/SNR = 0.1, 0.5, and 1.5 (training case). The DAE outperformed CF in denoising, especially in the high noise level case. DAE allowed the noise to be removed. The flow fields could be accurately reconstructed. Figure 7 displays the root-mean-square profiles of velocity fields sampling at various positions of $x$ axis. The results obtained from DAE were consistent with those of DNS. However, the results of the CF exhibit apparent deviations compared with DNS. Figure 8 displays the relative error of training, interpolation, and extrapolation cases. The relative errors revealed an increasing trend with the increase in the noise level. However, all the errors were minor, where the maximum error of $u$ is 0.05% and 1.5% for $v$. Although the interpolation case never attends DAE training, the errors of the interpolation case were similar to those of the training case at 1/SNR =1.5. Notably, the extrapolation case not only absented in DAE training but also featured noise levels beyond the range encountered in the training cases. However, the extrapolation case could be effectively denoised, resulting in a low error level. Therefore, Figure. 8 indicates that the proposed denoising methods exhibit excellent generalization that is suitable for denoising noisy data at a range of noise levels, even if the noise level is out of the range of the training case.

In this subsection, noisy bluff body flow data with various noises, that is, Gaussian noise, salt-and-pepper noise, and speckle noise at the same noise level (1/SNR = 0.5), were passed to train the DAE, where each noisy data includes 1200 snapshots.

Figure 9 displays the denoise instantaneous contours of the bluff body flow, contaminated with three types of noise at a noise level of 1/SNR = 0.5. The results denoised by DAE appear smoother than those processed by CF. Moreover, although the CF struggled with denoising salt-and-pepper and speckle noise, DAE consistently removed these three distinct noise types. Furthermore, as depicted in Figure 10, the statistical outcomes derived from the DL method exhibit higher accuracy than those obtained through CF.

In this subsection, DMDAE was used to denoise turbulent channel flows at $Re_{\tau}$ = 180, 395, and 550, which presents a higher level of complexity than the laminar flow cases. The dataset used for training DMDAE comprises cases of noisy channel flow data with Gaussian noise at 1/SNR = 0.5, each containing 1200 snapshots. Various outcomes, including instantaneous contours and turbulent statistics, are illustrated to demonstrate that the denoise data numerically and physically align with DNS data. The instantaneous contours (Figure 11) indicate that DMDAE effectively denoises and reconstructs the three-channel flow cases. By contrast, the CF method could not accurately distinguish flow features from noise, erroneously eliminating flow features as noise. Figures 12, 13, and 14 demonstrate the velocity and vorticity RMS profiles for the three-channel flow cases. The outcomes obtained from DMDAE were consistent with the DNS results, whereas the results from CF exhibited deviations compared with DNS. Figure 15 displays the probability density function ($p.d.f$) of the velocity components for channel flow. The velocity distribution curves of the DL results were consistent with the ground truth. Furthermore, the most pronounced deviation in the CF results occurred in the case of $Re_{\tau}$ = 550, where the flow was more chaotic and challenging to denoise. Figures16, 17, and 18 reveal the spanwise energy spectra of the velocity components. Energy spectra results revealed that conventional denoising filters perform poorly on turbulent flow denoising as the eddies of channel flow were mistakenly eliminated. The proposed DL method performs better in ensuring that the denoise results are consistent with the DNS data.

In this subsection, the denoising performance of the DAE was examined using noisy PIV data. Figure 19 displays the denoise instantaneous contours of the PIV data. Comparing the contours of denoise and DNS data reveals that the proposed DL model can remove most noise from noisy PIV data. However, the CF cannot denoise sufficiently where some large-scale noise exists in the flow field. By contrast, DAE cannot remove the shadow region near the bluff body. The shadow region exists in every snapshot of the PIV data; therefore, DAE regards the shadow as a part of the flow and retains it. Thus, DAE can only remove the noise randomly appearing but not restore the permanent corruption within the flow field. Figure 20 represents the RMS profiles of the velocity components, where a DNS case having the same $Re_{d}$ as the PIV data is also plotted. The figure reveals that the denoised data are closer to the DNS results after denoising using the DL model.