Evaluating Feature Attribution Methods for Electrocardiogram

Feature attribution is a technique that measures the importance of each pixel in an image for a given prediction of a model. Because feature attribution values can be easily visualized as a heatmap showing which pixels are important, it has become popular in the image domain [2]. Feature attribution methods can be categorized according to the mechanisms they use [2, 6]. Backpropagation-based methods use gradients of a model computed by the backpropagation process to measure the importance of each element in the input. This group includes Saliency [7], Input$\times$Gradient [8], Guided Backprop [9], Integrated Gradients [10], DeepLIFT [8], DeepSHAP [11] and LRP [12]. Perturbation-based methods measure feature attributions by tracking the model outputs for the given perturbed inputs. This group includes LIME [13] and KernelSHAP [11]. Activation-based methods use activation maps where the importance for the target class is computed. This group includes Grad-CAM [14]. We also classify Guided Grad-CAM [14], which combines Grad-CAM and Guided Backprop, as an activation-based.

A series of studies have proposed evaluation metrics to analyze and compare feature attribution in the context of computer vision research [2]. Evaluation metrics examine whether each attribution method explains a model’s prediction with a high fidelity. They can be categorized mainly into two groups: localization-based metrics and perturbation-based metrics.

Localization-based metrics [7, 15, 16, 17, 18, 19] assess how successfully a feature attribution method localizes the input’s class-discriminative features. To evaluate feature attribution methods using localization-based metrics, we need a dataset with additional ground-truth information on which area of an input is closely related to the label. For this reason, researchers in the image domain have conducted experiments with special datasets containing object bounding boxes or have constructed image grids [6]. As localization-based metrics utilize human-made labels, they reflect human knowledge where a model should attend to.

Perturbation-based metrics [16, 20, 21] assess feature attribution methods by perturbing an input and measuring how much the prediction of a model is affected. If an attribution method is effective, its heatmap should match the areas that are sensitive to the perturbation. An advantage of perturbation-based metrics is that they do not require additional ground-truth information. Therefore, perturbation-based metrics are applicable to a wider variety of datasets than localization-based metrics.

Feature attribution methods and evaluation metrics for them have been mainly developed in the computer vision research. The results from [2] imply that the performance of attribution methods should be measured for each specific use case. Thus, to apply feature attribution methods to ECG domain, such methods should be verified using ECG data and tasks. There are a limited number of studies that focus on evaluating feature attribution methods using ECG data [3, 4, 22]. However, the evaluation metrics used in the previous studies are designed for general time-series data based on shapelets [3] or perturbation-based metrics [3, 4, 22]. Moreover, the number of feature attribution methods studied in the three studies is at most six.

A sound evaluation can be achieved by concurrently evaluating additional feature attribution methods with customized metrics that reflect ECG-specific characteristics. In this study, we propose two localization-based metrics that utilize the pseudo-periodic structure of an ECG signal and one perturbation-based metric that jointly considers the performance drop under the increasing and decreasing order of attribution values. We compare eleven feature attribution methods with our proposed evaluation metrics, which is roughly twice of the previous studies.

Localization Score is a metric that measures the overlap between the area with high attribution values and the ground truth area [7, 15, 16]. Considering the characteristics of the ECG data, we propose the following metric for calculating the localization score. If the abnormal beats of an example contain $n$ data points, we select top-$n$ data points with the highest attribution values. Then we calculate the Intersection-over-Union (IoU) between the selected $n$ data points and the $n$ data points in abnormal beats. IoU between two sets $A$ and $B$ is defined as follows:

Pointing Game is a metric that measures if a data point with the maximum attribution value correctly points to the ground truth area [17, 18, 19]. This metric is different from the localization score in that the pointing game does not require highlighting the full extent of an object and only focuses on the maximum attribution point [17]. We measure the pointing game accuracy as follows: If a data point with the highest attribution value falls on one of the abnormal beats, it counts as a hit; otherwise, it counts as a miss. Then we calculate the accuracy as described in [17].

Degradation Score is a metric that examines how a model’s prediction degrades under cumulative perturbation of an original input, suggested by [16]. Perturbations can be applied to small-sized windows of the input in the order of decreasing attribution values (Most Relevant removed First, MoRF) [20] or of increasing attribution values (Least Relevant removed First, LeRF) [23]. The degradation score combines both perturbation processes. We scale the model’s output probability for a correct class of an original example to one and the probability of a fully perturbed example to zero. Then we plot MoRF and LeRF curves as shown in the lower right side of Figure 1 and compute the integral between the two curves. As a perturbation, we use a mean operation; we fill a selected window with the mean value of the window. We select the mean operation since it can erase detailed wave information while preserving the typical shape of an ECG signal.

Dataset. We experimented with the Icentia11K dataset [24, 25]. Icentia11K is one of the largest public ECG datasets containing 11 thousand patients and 2 billion labeled beats from a single-lead ECG device. We mostly followed the experimental settings of the original Icentia11K beat classification task [24]. The goal of the beat classification task is to predict if an example consists of all normal beats or contains at least one premature atrial contraction (PAC) or premature ventricular contraction (PVC). Each ECG signal has 2049 data points that correspond to about 8 seconds, as the sampling rate of the dataset is 250Hz. We use R-peak locations provided in the dataset to extract beats by using the midpoint of two adjacent R-peaks as the endpoint of a beat as done in [26].

Training and Evaluation. Each of the training and test sets contain 6000 examples - 2000 for normal, PAC, and PVC classes respectively. When evaluating feature attribution methods with the proposed metrics, we only used test examples correctly predicted as PAC or PVC with over 90% probability. The reason for using only PAC and PVC examples for evaluation is that the localization-based metrics cannot be measured on an example labeled as normal since it consists of all normal beats, and all beats are true localization targets. Moreover, the reason for using only correctly predicted examples with high probability is to ensure that the low evaluation result of feature attribution methods is attributable to the attribution method and not the classification model’s poor performance [19]. The total number of examples used to compute evaluation metrics is around 3272 on average.

Implementation Details. Since the original ResNet [27] was created for 2D images, we build a modified 18-layer ResNet with $1$x$7$-sized kernels for designed for 1D ECG signals. Each ECG signal of 2049 points is standardized to have zero mean and one standard deviation. The model is trained with cross-entropy loss and Adam optimizer. We set learning rate to 5e-4, batch size to 128, weight decay to 1e-7, and training epochs to 50. The classification accuracy of our model on the test set is reported in Table 1, which exceeds that of previous models evaluated on the same dataset. For evaluating the degradation score, we set the perturbation window size to 16. All results are the average of 5 repeated experiments with different dataset splits and model initializations.

The results are shown in Table 2. Among the eleven attribution methods, Grad-CAM achieves the best performance across all three evaluation metrics. For the localization score, the two activation-based methods, including Grad-CAM, significantly outperform all of the backpropagation-based and perturbation-based methods. For the pointing game, only Integrated Gradients performs comparably with Grad-CAM. For the degradation score, four of the backpropagation-based methods perform comparably with Grad-CAM. Overall, the average score of Grad-CAM is outstanding where it is 18% better than the second-best method, Integrated Gradients. The visualization is provided in Figure 3, where the three best methods from each category are shown.

An extensive evaluation of feature attribution methods in the image domain can be found in [2]. Compared to the image domain, ECG domain turns out to be distinct in a few ways. For instance, the two perturbation-based methods, LIME and KernelSHAP, perform reasonably well in the image domain but poorly in the ECG domain. DeepSHAP, which is known to achieve a competitive performance to Grad-CAM on ImageNet [2], also performs poorly in the ECG domain except for the degradation score. Grad-CAM, however, turns out to perform well for both domains. As confirmed in [2], different attribution metrics seem to measure different underlying concepts. For ECG domain, we can conclude that Grad-CAM is an appropriate feature attribution method that measures underlying concepts that are strongly relevant to cardiac arrhythmia detection.