PAGE: Domain-Incremental Adaptation with Past-Agnostic Generative Replay for Smart Healthcare

The evolution of WMSs and ML has enabled efficient, effective, and accessible disease-detection systems. For example, Alfian et al. [2] propose a personalized healthcare monitoring system based on Bluetooth Low Energy (BLE)-based sensor devices and ML classifiers for diabetic patients. They gather distinct vital sign data from users with commercial BLE-based sensors. Then, the system analyzes the data with a multilayer perceptron (MLP) for early diabetes detection and a long short-term memory for future blood glucose level prediction. Farooq et al. [8] introduce a cost-effective electrocardiography (ECG) sensor to obtain ECG waveforms from patients and use K-means clustering to classify various arrhythmia conditions. Shcherbak et al. [36] exploit commercial WMSs to collect motion data using designated exercises from test subjects and use ML classifiers to detect early-stage Parkinson’s disease. Himi et al. [15] construct a framework, called MedAi, that adopts a prototype smartwatch with eleven sensors to collect physiological signals from users and a random forest (RF) algorithm to predict numerous common diseases. Wang et al. [44] develop an RF-based diagnostic system to detect the sleep apnea syndrome by collecting human pulse wave signals and blood oxygen saturation levels with a photoplethysmography optical sensor. Hassantabar et al. propose CovidDeep [11] and MHDeep [13] to detect the SARS-CoV-2 virus/COVID-19 disease and mental health disorders based on physiological signals collected with commercial WMSs and smartphones. They employ a grow-and-prune algorithm to synthesize optimal deep neural network (DNN) architectures for disease detection. Similarly, Yin et al. build DiabDeep [49] with grow-and-prune synthesis to deliver DNN models with sparsely-connected layers or sparsely-recurrent layers for diabetes detection based on WMS data and patient demographics.

ML models suffer from performance degradation when presented with data from outside of the underlying domains they were trained on. Simply fine-tuning these models with new data results in CF. Therefore, researchers have been working on developing various algorithms to create more robust and general models. Domain adaptation [7, 10, 5] is a sub-field in ML research that aims to address the aforementioned problem. However, most works in this sub-field focus on making ML models generalize from a source domain to a target domain. They are inapplicable to incremental multi-domain adaptation scenarios. Hence, we turn to CL strategies that enable ML models to perform domain-incremental learning (adaptation) in real-world scenarios with nonstationary data domains. Numerous CL algorithms have been proposed in the literature. Recent CL algorithms can mainly be categorized into three groups: regularization-based, architecture-based, and replay-based [30, 31, 24, 6, 26, 21].

A function that describes the probability distribution of a continuous random variable is called a probability density function (PDF). It provides information about the likelihood of the random variable taking on a value within a specific interval. Probability density estimation methods are used to estimate the PDF of a continuous random variable based on observed data. These methods can be classified into two groups: parametric and non-parametric. In an ablation study presented in Section 6.3.1, we show that a parametric density estimation method performs better in our domain-incremental adaptation experiments with WMS data. Therefore, we implement the parametric density estimation method in PAGE.

Parametric density estimation assumes that a PDF belongs to a parametric family. Therefore, density estimation is transformed into estimating various parameters of the family. One common example of parametric density estimation is the Gaussian mixture model (GMM). Suppose there are $N$ independent and identically distributed (i.i.d.) samples $x_{1},x_{2},...,x_{N}$ drawn from an unknown univariate distribution $f$ at any given point $x$. The estimated PDF of $f$ is represented as a mixture of $C$ Gaussian models in the form:

CP was first proposed in [33, 42]. It complements each prediction result with a measure of confidence and credibility. The confidence score gives a measurement of the model uncertainty on the prediction. The credibility value indicates how suitable the training data are for producing the prediction. The paradigm can adopt any ML algorithm as the prediction rule, such as a support-vector machine, ridge regression, or a neural network [35]. By measuring how unusual a prediction result is relative to others, CP provides a statistical guarantee to a prediction through the confidence score and credibility value.

The original CP uses transductive inference to generate prediction with the underlying ML algorithm and all seen samples. However, it is highly computationally inefficient. Therefore, an alternative CP method based on inductive inference was proposed to address this issue [29, 27, 28]. Here, we give an overview of the main concepts behind the transductive conformal prediction (TCP) and inductive conformal prediction (ICP) methods. For a more comprehensive description and statistical derivations, see [35, 43].

CL is defined as training ML models on a series of tasks where non-stationary data domains, different classification classes, or distinct prediction tasks are presented sequentially [31, 46]. Therefore, CL problems are usually categorized into three scenarios: domain-, class-, and task-incremental learning [41]. In this work, we focus on the domain-incremental learning scenario for our disease-detection tasks.

Domain-incremental adaptation can be defined as the scenario where data from different probability distribution domains for the same prediction task become available sequentially. For example, data collected from different cities, countries, or age groups may become available incrementally for the same disease-detection task. Given a prediction task, we define a potentially infinite sequence of various domains as $D=\{D_{1},D_{2},...,D_{n},...\}$, where the $n$-th domain is depicted by $D_{n}=\{(X_{n}^{i},Y_{n}^{i})~{}|~{}i=1,2,...,C\}$. $X_{n}^{i}\in\mathcal{X}$ and $Y_{n}^{i}\in\mathcal{Y}$ refer to the set of data features and their corresponding target labels for the $i$-th class in domain $D_{n}$. $C$ refers to the total number of classification classes for the prediction task. The objective of domain-incremental adaptation is to train an ML model $f_{\theta}:\mathcal{X}\rightarrow\mathcal{Y}$, parameterized by $\theta$, such that it can incrementally learn new data domains in $D$ and predict their labels $Y=f_{\theta}(X\in\mathcal{X})\in\mathcal{Y}$ without degrading performance on prior domains.

Several metrics are commonly used to evaluate CL algorithms in the literature, including average accuracy, average forgetting, backward transfer (BWT), and forward transfer [31, 21]. We report the average accuracy and average F1-score across all learned domains in our experimental results to validate PAGE’s effectiveness in domain-incremental adaptation for various disease-detection tasks. When reporting the average F1-score, we define true positives (negatives) as the unhealthy (healthy) instances correctly classified as disease-positive (healthy) and false positives (negatives) as the healthy (unhealthy) instances misclassified as disease-positive (healthy). In addition, we report BWT to evaluate how well PAGE mitigates CF. BWT measures how much a CL strategy impacts a model’s performance on previous tasks when learning a new one. For a model that has learned a total of $q$ tasks, BWT can be defined as:

First, we provide an overview of the proposed PAGE strategy. Fig. 1 shows its schematic diagram. It is composed of three steps for performing domain-incremental adaptation and generating system outputs. For example, we use PAGE to learn a new data domain $D_{n}$ with an ML model that has learned $n-1$ domains for a disease-detection task using WMS data. PAGE passes the new real WMS data from $D_{n}$ through an SDG module to generate synthetic data that retains the knowledge learned from domains $D_{1}$ to $D_{n-1}$. Next, the synthetic data are replayed together with the new real data to update the model in a generative replay fashion. Finally, PAGE uses the proposed EICP method to produce detection results and their corresponding confidence scores and credibility values.

Fig. 2 shows the top-level flowchart of the three steps of PAGE. In Step 1, as shown in Fig. 2(a), the SDG module exploits only the real training and validation data from the new domain to produce synthetic data for training and validation, respectively. In Step 2, depicted in Fig. 2(b), the synthetic data for training are replayed with the real training data from the new domain to update the model. During this process, PAGE also records the average training loss values of all training data for Step 3. In Step 3, illustrated in Fig. 2(c), EICP uses the real validation data from the new domain, the synthetic data generated for validation, the average training loss values, and the new data for detection as inputs. Finally, EICP generates detection results along with their corresponding confidence scores and credibility values to provide model interpretability and statistical guarantees to users.

Next, we dive into the details of each step of PAGE.

The SDG module generates synthetic data that include pairs of synthetic data features and pseudo labels for replay, to retain knowledge of learned domains. We draw inspiration from a framework called TUTOR [12] to build our SDG module. First, the module models the joint multivariate probability distribution of the real data features from the new domain through a probability density estimation function. Next, it generates synthetic data features by sampling from the learned distribution. Then, it passes the synthetic data features to the current model and records the responses from the output softmax layer. It assigns these raw output probability distributions as pseudo labels of the synthetic data features. The pairs of synthetic data features and their pseudo labels effectively represent the knowledge of the previous domains that the model has learned so far. Therefore, we can replay the synthetic data during the model update process to retain the learned knowledge.

We adopt a multi-dimensional GMM as the probability density estimation function in our SDG module. The probability distribution of the real data features is modeled as a mixture of $C$ Gaussian models, as mentioned in Section 2.3. The PDF of the learned distribution can be expressed using Eq. (1). Note that previous works [11, 13, 12, 40] have already shown that GMM can recreate the probability distribution of real data and generate high-quality synthetic data. The generated synthetic data can then replace real data in the training process while achieving similar test accuracy. However, one drawback of this method lies in its learning complexity. To address this problem, we use the expectation-maximization (EM) algorithm to determine the GMM parameters [12]. The EM algorithm iteratively solves this optimization problem in the following two steps until convergence:

• •

  Expectation-step: Given the current parameters $\theta_{i}$ of the GMM and observations $X$ from the real training data, estimate the probability that $X$ belongs to each Gaussian model component $c$.

• •

  Maximization-step: Find the new parameters $\theta_{i+1}$ that maximize the expectation derived from the expectation-step.

Algorithm 1 shows the pseudocode of the SDG module. It is important to note that the input real data features for training GMM $X_{train\_gmm}$ and the input real data features for validating GMM $X_{valid\_gmm}$ refer to the real data features that are used to train and validate the underlying GMM, respectively. They do not stand for the real training data features $X_{train}$ and real validation data features $X_{valid}$ from the new domain. The algorithm starts by initializing a set $\mathcal{C}$ of Gaussian model components, ranging from 1 to a user-specified maximum number $C_{max}$. Then, for each candidate number $C\in\mathcal{C}$, the algorithm fits a GMM $gmm_{C}$ to the given $X_{train\_gmm}$ with $C$ Gaussian model components. To determine the optimal total number of components $C^{*}$, we use the Bayesian information criterion (BIC) [34] for model selection. BIC is a commonly used criterion in statistics to measure how well a model fits the underlying data. BIC favors models that have a good fit with fewer parameters and smaller complexity, which results in a smaller chance of overfitting [39]. The algorithm computes the BIC score bic() of each GMM $gmm_{C}$ to evaluate the quality of the model. To further avoid the risk of overfitting, we use this criterion with the given $X_{valid\_gmm}$. A lower BIC score implies a better fit in model selection. Therefore, the number $C$ that minimizes this criterion is chosen as the optimal $C^{*}$. This means that $C^{*}$ Gaussian models are required to model the probability distribution of $X_{train\_gmm}$. Then, the algorithm finalizes the optimal GMM $gmm^{*}$ with $C^{*}$ components to fit $X_{train\_gmm}$. Next, it samples a user-defined number $count$ of synthetic data features $X_{syn}$ from $gmm^{*}$. Finally, we pass $X_{syn}$ to the current model $\mathbf{M}$ and assign the raw output probability distributions from its softmax output layer as their pseudo labels $Y_{syn}$.

We repeat this procedure twice to generate synthetic data for training and validation. When we generate the synthetic data for training $(X_{syn\_train},Y_{syn\_train})$, the inputs are $X_{train\_gmm}=X_{train}$ and $X_{valid\_gmm}=X_{valid}$. When we generate the synthetic data for validation $(X_{syn\_valid},Y_{syn\_valid})$, the inputs are $X_{train\_gmm}=X_{valid}$ and $X_{valid\_gmm}=X_{train}$.

In this step, we update the model and record the average training loss values of all the training data, real and synthetic, for the next step. To update the model while mitigating CF, we replay the synthetic training data with the real training data from the new domain during training. Due to the difference in dataset sizes between the real and synthetic training data, the model might be biased toward the dataset with more data. To avoid the biased prediction problem, we sample the same number of data instances from each dataset to form each mini-batch to train the model. In other words, if we set the mini-batch size to $B$, $\frac{B}{2}$ of the data are sampled from the real training data from the new domain while the other $\frac{B}{2}$ of the data are sampled from the synthetic training data.

Algorithm 2 demonstrates the pseudocode of the model update procedure. In the beginning, we initialize a vector $\mathcal{L}$ to store the average training loss of each data instance used in the training process. In addition, we concatenate the real and synthetic validation data features ($X_{valid},X_{syn\_valid}$) and their labels ($Y_{valid},Y_{syn\_valid}$) to prepare validation data ($x_{valid}$, $y_{valid}$) for model update. Then, we train the model for $E$ epochs with training data in a mini-batch size $B$. Within each epoch, while data instances are not exhausted in both real training data ($X_{train}$, $Y_{train}$) and synthetic training data ($X_{syn\_train}$, $Y_{syn\_train}$), we sample $\frac{B}{2}$ data instances from ($X_{train}$, $Y_{train}$) and another $\frac{B}{2}$ from ($X_{syn\_train}$, $Y_{syn\_train}$) to form a mini-batch data ($x_{train}$, $y_{train}$) to update the model parameters $\theta$. For each data instance in ($x_{train}$, $y_{train}$), we record the training loss value in a temporary vector $v$. At the end of each epoch, we accumulate the training loss values of all training data instances in vector $\mathcal{L}$. Also, we store the current $\theta$ and the resulting performance on ($x_{valid}$, $y_{valid}$). After updating the model for $E$ epochs, we assign the $\theta$ that maximizes performance on ($x_{valid}$, $y_{valid}$) to the updated model $\mathbf{M}_{new}$. Finally, we obtain the average of the training loss values over $E$ epochs to retrieve the vector of average training loss values $\mathcal{L}$ for all training data.

Next, we propose EICP, an extended version of the original ICP, to generate a detection result, confidence score, and credibility value for a new data instance. The new data instance can belong to any domain that the model has learned so far. Similar to the original ICP introduced in Section 2.4.2, we combine the real and synthetic training data into a proper training set to train the underlying ML model. Next, we use the real and synthetic validation data as a calibration set to calculate non-conformity scores. As opposed to ICP, we extend the calibration set by including a subset of the proper training set. By doing so, we strengthen the ability of the calibration set to perform CP based on the learned domains.

Fig. 3 shows the flowchart of EICP. It consists of three processes. In Process 1, EICP selects a subset of data from all training data, including the real and synthetic training data, to form the extended calibration set. As shown in Fig. 3(a), EICP selects the data through a data selection (DS) module based on their average training loss values. In Process 2, EICP computes the non-conformity scores of the extended calibration set through a non-conformity measure. As shown in Fig. 3(b), the softmax output responses of the extended calibration set from the updated model are used to compute their non-conformity scores. In Process 3, EICP first retrieves the softmax output response of the new data instance from the updated model and computes its non-conformity score using the non-conformity measure. Finally, as illustrated in Fig. 3(c), EICP produces the detection result for the new data instance, confidence score, and credibility value using the non-conformity scores of the new data instance and the calibration set through a p-value function, as described in Section 2.4.2.

We have chosen three different disease datasets collected from commercial WMSs to evaluate PAGE’s effectiveness in domain-incremental adaptation for disease detection. These are the CovidDeep [11], DiabDeep [49], and MHDeep [13] datasets from the literature. Data collection and experimental procedures for these datasets were governed by Institutional Review Board approval. The efficacy of these datasets has been validated by results presented in the literature [11, 49, 13, 19].

Table 1 shows the data features collected in the CovidDeep dataset. It contains various continuous physiological signals and Boolean responses to a simple questionnaire. The dataset was collected from 38 healthy individuals, 30 asymptomatic patients, and 32 symptomatic patients at San Matteo Hospital in Pavia, Italy. The data were acquired using commercial WMSs and devices, including an Empatica E4 smartwatch, a pulse oximeter, and a blood pressure monitor. Hassantabar et al. conducted numerous experiments [11] to identify which data features are the most beneficial to detecting COVID-19 in patients for obtaining a high disease-detection accuracy.

Table 2 shows the data features recorded in the DiabDeep dataset. It includes the physiological signals of participants and additional electromechanical and ambient environmental data. All data features have continuous values. The dataset was obtained from 25 non-diabetic individuals, 14 Type-I diabetic patients, and 13 Type-2 diabetic patients. While the physiological data were collected with an Empatica E4 smartwatch, other data were recorded using a Samsung Galaxy S4 smartphone. The additional smartphone data were proven to provide diagnostic insights through user habit tracking [48]. For instance, they help with body movement tracing and physiological signal calibration.

Table 2 also shows the data features recorded in the MHDeep dataset. It collects the same physiological signals and additional data features as those collected in the DiabDeep dataset. The dataset was obtained from 23 healthy participants, 23 participants with bipolar disorder, 10 participants with major depressive disorder, and 16 participants with schizoaffective disorder at the Hackensack Meridian Health Carrier Clinic, Belle Mead, New Jersey. As before, the data were collected using an Empatica E4 smartwatch and a Samsung Galaxy S4 smartphone.

First, we preprocess all datasets to transform them into a suitable format for our experiments. We synchronize and window the data streams by dividing data into 15-second windows with 15-second shifts in between to avoid time correlation between adjacent data windows. Each 15-second window of data constitutes one data instance. Next, we flatten and concatenate the data within the same time window from the WMSs and smartphones. Then, we concatenate the sequential time series data with responses to the questionnaire for the CovidDeep dataset. This results in a total of 14047 data instances with 155 features each for the CovidDeep dataset, a total of 20957 data instances with 4485 features each for the DiabDeep dataset, and a total of 27082 data instances with 4485 features each for the MHDeep dataset. Subsequently, we perform min-max normalization on the feature data in all datasets to scale them into the range between 0 and 1. This prevents features with a wider value range from overshadowing those with a narrower range.

Next, we perform dimensionality reduction on the DiabDeep and MHDeep datasets using principal component analysis to reduce their dimension from 4485 to 155. First, we standardize the feature data in the datasets and compute the covariance matrix of the features. Then, we perform eigendecomposition to find the eigenvectors and eigenvalues of the covariance matrix to identify the principal components. Next, we sort the eigenvectors in descending order based on the magnitude of their corresponding eigenvalues. Finally, we construct a projection matrix to select the top 155 principal components. Afterward, we recast the data along the principal component axes to retrieve the 155-dimensional feature data for these datasets.

Due to the limited amount of patient data available in the datasets, we assume two different data domains in our domain-incremental adaptation experiments. For each dataset, we first randomly split the patients into two domains in a stratified fashion, where Domain 1 has 80% of the patients from each class and Domain 2 has the remaining 20%. Next, within each domain, we take the first 70%, the next 10%, and the last 20% of each patient’s sequential time series data to construct the training, validation, and test sets with no time overlap. Finally, we apply the Synthetic Minority Oversampling Technique (SMOTE) [3] to the partitioned training datasets to counteract the data imbalance issue within each dataset. We start by selecting a random data instance $A$ in a minority class and finding its five nearest neighbors in that class. Then, we randomly select one nearest neighbor $B$ from the five and draw a line segment between $A$ and $B$ in the feature space. Finally, we generate a synthetic data instance at a randomly selected point on the line segment between $A$ and $B$. We then repeat this process until we obtain a balanced number of data instances in all classes in each partitioned training set.

PAGE is applicable to any generic DNN model that performs classification on sequential time series data with a softmax output layer. For experimental simplicity, we use an MLP model to evaluate our PAGE strategy. We use grid search to obtain the best hyperparameter values for our MLP model, using the validation sets of the three datasets. We find that a four-layer architecture provides the optimal performance, in general. Fig. 5 shows the architecture of the final MLP model. It has an input layer with 155 neurons to align with the input dimensionality of the datasets. Subsequently, it has three hidden layers with 256, 128, and 128 neurons in each layer, respectively. Finally, it has an output layer with the number of neurons that corresponds to the number of possible classes. We use the rectified linear unit (ReLU) as the nonlinear activation function in the hidden layers and the softmax function for the output layer to generate the output probability distributions.

We use the stochastic gradient descent optimizer with a momentum of 0.9 in our experiments and initialize the learning rate to 0.005. We set the batch size to 128 for training, where data are drawn evenly from the real and synthetic training data, as described in Section 4.3. We train the MLP model for 300 epochs. We set the maximum number of Gaussian model components ($C_{max}$) for the SDG module to 10. In addition, based on the ablation study described in Section 6.3.2, we set the number of synthetic data instances ($count$) to 90% of the number of real data features for training GMM ($X_{train\_gmm}$) in the new domain. In other words, the number of synthetic training (validation) data instances is 90% of the number of the new real training (validation) data instances. Similarly, based on the ablation study described in Section 6.3.3, we set the upper-level percentile ($p_{upper}$) to the 90th percentile and the lower-level percentile ($p_{lower}$) to the 70th percentile of the average training loss values of each classification class in the DS module in EICP. Finally, we set the parameter $\gamma$ in Eq. (3) to 2.0 for our non-conformity measure.

We implement PAGE with PyTorch and perform the experiments on an NVIDIA A100 GPU. We employ CUDA and cuDNN libraries to accelerate the experiments.

First, we evaluate the performance of PAGE for domain-incremental adaptation on the datasets introduced in Section 5.1. We compare PAGE against five different strategies, including naive fine-tuning, ideal joint-training, DOCTOR [19], EWC [16], and LwF [20]. The naive fine-tuning method naively fine-tunes the MLP model with real training data from new domains without adopting any CL algorithm. The ideal joint-training method represents the ideal scenario where we have access to all training data from both past and new domains to train the MLP model with all data jointly. DOCTOR performs replay-style CL by either preserving raw training data from past domains or using data from previous domains to generate synthetic data for future replays. We report its best performance based on its two algorithms. EWC stores a Fisher information matrix for learned domains to weigh the importance of each model parameter. Then, it imposes a quadratic penalty on the update process of the important parameters. LwF distills the knowledge of learned domains with the pseudo labels given to the real training data from new domains. Then, it trains the model on the new real training data with both their pseudo and ground-truth labels.

Table 3 presents the domain-incremental adaptation experimental results for all three datasets. The results are reported on the real test data from both domains of all datasets. As shown in the first row for each dataset, the MLP model without domain adaptation performs poorly on data from the second domain and has a low average test accuracy. The naive fine-tuning method makes the MLP model overfit to the second domain and performs poorly on the first domain due to CF. PAGE outperforms the naive fine-tuning method and achieves superior results compared to EWC and LwF. While DOCTOR and the ideal joint-training method perform very well on domain-incremental adaptation and achieve high average test accuracy, they require a memory buffer that scales up with the number of new domains to preserve data from all seen domains. On the other hand, PAGE achieves very competitive results with negligible loss in average test accuracy without incurring the memory buffer cost and suffering from scalability issues.

At test time, users can input their pre-processed WMS data into PAGE and obtain the disease-detection results, confidence scores, and credibility values, as shown in Fig. 1. The confidence scores and credibility values can help users determine if their detection results are robust enough and whether clinical intervention is required. For healthy results with high confidence scores and credibility values, clinical intervention is not necessary. For disease-positive results with high confidence scores and credibility values, users can be alerted to seek medical treatments.

To demonstrate and evaluate the performance of EICP in PAGE against the original ICP, we report their performances in a confusion matrix format. Tables 4, 5, and 6 show the CP experimental results for the CovidDeep, DiabDeep, and MHDeep datasets, respectively. The results are reported on the real test data from both domains of all three datasets. The rows of the confusion matrices indicate whether the model is certain or uncertain about its detection results. We say that the model is certain when its confidence score and credibility value are higher than certain thresholds. We select the threshold values for each dataset based on their validation datasets. The confidence score threshold for all three datasets is 90%. The credibility value thresholds for the CovidDeep, DiabDeep, and MHDeep datasets are 70%, 85%, and 90%, respectively. The columns indicate whether the detection results are correct or incorrect. The detection result is correct when it correctly predicts the ground-truth label. We define two metrics to evaluate CP performance: correctness and error rate. Correctness is defined as the number of certain and correct detection results over the number of all test data instances. Error rate is defined as the number of certain but incorrect detection results over the number of all test data instances.

As shown in the tables, EICP achieves superior correctness compared to the original ICP with an identical error rate for both CovidDeep and DiabDeep datasets. For MHDeep, EICP outperforms ICP in correctness with only a 0.3% increase in the error rate for Domain 1. These results demonstrate that EICP reflects model uncertainty more precisely than ICP because of its smaller number of correct but uncertain results. EICP also provides a more accurate statistical guarantee for disease detection (correct and certain). In addition, the results show that EICP reduces clinical workload by up to 75% (based on the correctness metric).

Next, we present the results of some ablation studies.