An Accurate Non-accelerometer-based PPG Motion Artifact Removal Technique using CycleGAN

For the clean dataset, we use BIDMC dataset (Pimentel et al., 2016). This dataset contains signals and numerics extracted from the much larger MIMIC II matched waveform database, along with manual breath annotations made from two annotators, using the impedance respiratory signal.

The original data was acquired from critically ill patients during hospital care at the Beth Israel Deaconess Medical Centre (Boston, MA, USA). Two annotators manually annotated individual breaths in each recording using the impedance respiratory signal. There are 53 recordings in the dataset, each being 8 minutes long and containing:

• •

  Physiological signals, such as the PPG, impedance respiratory signal, and electrocardiogram (ECG) sampled at 125 Hz.

• •

  Physiological parameters, such as the heart rate (HR), respiratory rate (RR), and blood oxygen saturation level (SpO2) sampled at 1 Hz.

• •

  Fixed parameters, such as age and gender.

• •

  Manual annotations of breaths.

We conducted laboratory-based experiments to collect accelerometer data for generating noisy PPG signals. Each of these laboratory-based experiments consisted of 27 minutes of data. A total of 33 subjects participated in the laboratory-based experiments. In each experiment, subjects were asked to perform specific activities while the accelerometer data were collected from them using an Empatica E4 (cit, [n.d.]) wristband worn on their dominant hand. The Empatica E4 wristband is a medical-grade wearable device that offers real-time physiological data acquisition, enabling researchers to conduct in-depth analysis and visualization. Figure 2 shows our experimental procedure. Note that the accelerometer signals are only required for generating/emulating noisy PPG signals, and our proposed motion artifact removal method does not depend on having access to acceleration signals.

According to Figure 2, each experiment consists of 6 different activities: (1) Finger Tapping, (2) Waving, (3) Shaking Hands, (4) Running Arm Swing, (5) Fist Opening and Closing, and (6) 3D Arm Movement. Each activity lasts 4 minutes in total, including two parts with two different movement intensities (low and high), each of which lasts 2 minutes. Activity tasks are followed by a 30 seconds rest (R) period between them. During the rest periods, participants were asked to stop the previous activity and put both their arms on a table, and stay in a steady state. Accelerometer data collected during each of the activities were later used to model the motion artifact. We describe this in the next subsection.

To generate noisy PPG signals from clean PPG signals, we use accelerometer data collected in our study. Clean PPG signals are directly collected from the BIDMC dataset. We down-sample these signals to 32 Hz to ensure they are synchronized with the collected accelerometer data.

Empatica has an onboard MEMS type 3-axis accelerometer that measures the continuous gravitational force (g) applied to each of the three spatial dimensions (x, y, and z). The scale is limited to $\pm 2$g. Figure 3 shows an example of accelerometer data collected from E4.

Along with the raw 3-dimensional acceleration data, Empatica also provides a moving average of the data. Figure 4 visualizes the moving averaged data.

The following formula is used to calculate the moving averaged of the data,

in which, $\text{Curr}_{i}$ and $\text{Prev}_{i}$ are respectively the current value and the previous value of the accelerometer sensor (g) along the $i$-th dimension. The $\max(a,b,c)$ function returns the maximum value among $a$, $b$, and $c$.

Then the following formula is used to filter the output:

The filtered output (Avg) is directly used as a model for motion artifacts in our study. To simulate the noisy PPG signals, we add this noise model to a 2 minutes window of the clean PPG signals collected from the BIDMC dataset. We use 40 out of 53 signals in BIDMC directly as the clean dataset for training. Among these 40 signals, 20 are selected and augmented with the accelerometer data to construct the noisy dataset for training. The 13 remaining BIDMC signals and accelerometer data were added together to form the clean and noisy datasets for testing. In the rest of this section we describe each part of the model introduced in Figure 1.

To perform noise detection, first, the raw signal is normalized by a linear transformation to map its values to the range $(0,1)$. This can be performed using a simple function as below:

where $\text{Sig}_{\text{raw}}$ is the raw signal and $\text{Sig}_{\text{norm}}$ is the normalized output. Then, the normalized signal is divided into equal windows of size 256, which is the same window size we use for noise removal. These windows are then used as the input of the noise detection module to identify the noisy ones.

The similar type of machine learning network used in (Zargari et al., 2020) can be employed as a noise detection system. To explain the network structure for the noise detection method (Table 1 and Figure 5), first, we use a 1D-convolutional layer with 70 initial random filters with a size of 10 to select the basic features of the input data and convert the matrix size from $256\times 1$ to $247\times 70$. To extract more complex features from the data, another 1D-convolutional layer with the same filter size 10 is required. As the third layer, a pooling layer with a filter size of 3 is utilized. In this layer, a sliding window slides over the input of the layer and in each step, the maximum value of the window is applied to the other values. This layer converts a matrix size of $238\times 70$ to $79\times 70$. To select additional complex features, another set of convolutional layers are used with a different filter size. This set is followed by two fully connected layers of sizes 32 and 16. Lastly, a dense layer of size 2 with a softmax activation would produce the probability of each class: clean and noisy. The maximum of these two probabilities would be identified as the result of the classification. The accuracy of our proposed binary classification method is 99%, which means that the system can almost always detect a noisy signal from a clean signal.

In this section, we explore the reconstruction of noisy PPG signals using deep generative models. Once a noisy window is detected, it is sent to the noise removal module for further processing. First, the windows are transformed into 2-dimensional images, to exploit the power of existing image noise removal models, and then a trained CycleGAN model is used to remove the noise induced by the motion artifact from these images. In the final step of the noise removal, the image transformation is reversed to obtain the clean output.

The transformation needs to provide visual features for unexpected changes in the signal so that the CycleGAN model would be able to distinguish and hence reconstruct the noisy parts. To extend the 1-dimensional noise on the signal into a 2-dimensional visual noise on the image, we apply the following transformation:

where Sig is a normalized window of the signal. Each pixel will then have a value between 0 and 255, representing a grayscale image.

An example of such transformation is provided in Figure 6 for both the clean and the noisy signal. According to this figure, the noise effect is visually observable in these images.

Autoecnoders and CycleGAN are two of the most powerful approaches for image translation. These methods have proven to be effective in the particular case of noise reduction. Autoencoders require the pairwise translation of every image in the dataset. In our case, clean and noisy signals are not captured simultaneously, and their quantity differs. CycleGAN, on the other hand, does not require the dataset to be pairwise. Also, the augmentation in CycleGAN makes it practically more suitable for datasets with fewer images. Hence, we use CycleGAN to remove motion artifacts from noisy PPG signals and reconstruct the clean signals.

CycleGAN is a Generative Adversarial Network designed for the general purpose of image-to-image translation. CycleGAN architecture was first proposed by Zhu et al. in (Zhu et al., 2017).

The GAN architecture consists of two networks: a generator network and a discriminator network. The generator network starts from a latent space as input and attempts to generate new data from the domain. The discriminator network aims to take the generated data as an input and predict whether it is from a dataset (real) or generated (fake). The generator is updated to generate more realistic data to better fool the discriminator, and the discriminator is updated to better detect generated data by the generator network.

The CycleGAN is an extension of the GAN architecture. In the CycleGAN, two generator networks and two discriminator networks are simultaneously trained. The generator network takes data from the first domain as an input and generates data for the second domain as an output. The other generator takes data from the second domain and generates the first domain data. The two discriminator networks are trained to determine how plausible the generated data are. Then the generator models are updated accordingly. This extension itself cannot guarantee that the learned function can translate an individual input into a desirable output. Therefore, the CycleGAN uses a cycle consistency as an additional extension to the model. The idea is that output data by the first generator can be used as input data to the second generator. Cycle consistency is encouraged in the CycleGAN by adding an additional loss to measure the difference between the generated output of the second generator and the original data (and vice versa). This guides the data generation process toward data translation.

In our CycleGAN architecture, we apply adversarial losses (Goodfellow et al., 2014) to both mapping functions ($G:X\to Y$ and $F:Y\to X$). The objective of the mapping function $G$ as a generator and its discriminator $D_{Y}$ is expressed as below:

where the function $G$ takes an input from domain $X$ (e.g., noisy PPG signals), attempting to generate new data that look similar to data from domain $Y$ (e.g., clean PPG signals). In the meantime, $D_{Y}$ aims to determine whether its input is from the translated samples $G(x)$ (e.g., reconstructed PPG signals) or the real samples from domain $Y$. A similar adversarial loss is defined for the mapping function $F:Y\to X$ as $L_{GAN}(F,D_{X},Y,X)$.

As discussed before, adversarial losses alone cannot guarantee that the learned function can map an individual input from domain X to the desired output from domain $Y$. In (Zhu et al., 2017), the authors argue that to reduce the space of possible mapping functions even further, learned mapping functions ($Y$ and $F$) need to be cycle-consistent. This means that the translation cycle needs to be able to translate back the input from domain $X$ to the original image as $X\to G(X)\to F(G(X))\sim X$. This is called forward cycle consistency. Similarly, backward cycle consistency is defined as: $y\to F(y)\to G(F(y))\sim y$. This behavior is presented in our objective function as:

Therefore, the final objective of CycleGAN architecture is defined as:

where $\lambda$ controls the relative importance of the two objectives.

In Equation 7, $G$ aims to minimize the objective while an adversary $D$ attempts to maximize it. Therefore, our model aims to solve:

The architecture of the generative networks is adopted from Johnson et al. (Johnson et al., 2016). This network contains four convolutions, several residual blocks (He et al., 2016), and two fractionally-strided convolutions with stride $0.5$. For the discriminator networks, they use $70\times 70$ PathGANs (Li and Wand, 2016; Isola et al., 2017; Ledig et al., 2017).

After the CycleGAN is applied to the transformed image, the diagonal entries are used to retrieve the reconstructed signal.

In this section we compare our model’s efficiency with the state-of-the-art (Table 3). To minimize the difference between our experimental setup and the setups used in the related works we use the same measures. It should be noted that it is not feasible to perform a close comparison between our model and the existing works, due to the differences in the dataset and the lack of a public dataset providing noisy and clean signals simultaneously.

In comparison to non-accelerometer-based methods, our model significantly outperforms these models. The best performance observed in previous work is reported in (Hara et al., 2017) that improves the average RMSE from $28.26$BPM to $6.5$BPM ($4.3\times$ improvement). However, our model’s improvement on average RMSE is from $56.18$ to $2.18$ ($25.8\times$ improvement). In most of the existing accelerometer-based methods, no value is provided for the degree of the input noise. Although the best reported PPE belongs to (Raghuram et al., 2016) with an outcome of $0.392$BPM, the best improvement is achieved by (Lin and Ma, 2016) from $13.97$BPM to $6.87$BPM ($2.03\times$ improvement). However, our model’s improvement on average PPE is from $37.46$BPM to $0.95$BPM ($39.4\times$ improvement).