Optimising complexity of CNN models for resource constrained devices: QRS detection case study

Often, deep-learning (DL) based solutions stress to design a model complex enough, in terms of depth or other techniques, to understand a phenomena and achieve high performance for a given task. This is particularly valid for computer vision tasks [1, 2, 3], as well as, physiological time-series tasks [4, 5]. A deep model consists of a large number of layers, thus, a large number of parameters which require high computation capacity to train with a big enough dataset. The time and space complexity of such a deep model to deploy often exceeds the capacity of resource constrained environments (computing capacity, memory, energy etc.). The use of DL-based models in the client-side environment (sensor, smart-watch, or smart-phone) of IoT-based solutions, particularly, Internet of Medical Things (IoMT) based physiological signal monitoring solution are still unpopular and traditional filter-based approaches are commonly used [6, 7, 8, 9]. A shallow DL model may not be outstanding but has the capacity to offer a better solution together by leveraging the strengths of other essential components in an end-to-end IoMT solution architecture, which still lacks a rigorous exploration.

DL algorithms, as well as, filter-based classical methods which deal with physiological signals often require the input signal to pre-process and the algorithm’s output to post-process before a decision is finalised. The back-propagation training for DL models generally require the data to contain a certain-level of noise, in case where the data is clean, noise from external source is added to it so that the model is forced to be robust enough to learn sufficient characteristic features and be able to reduce over-fitting and better generalise over unknown test data [10, 11, 12]. This characteristic of noise robustness, in particular, is crucial for tasks with physiological signal which inherently contaminated with noise from multiple sources where threshold-based classical methods commonly fail to generalise over unknown test data. Classical filter-based methods in QRS-detection literature, for example, rarely report performance with multiple ECG datasets [13, 14, 15, 16, 17], whereas, recent DL approaches were found to test models across a broad range of datasets [5, 18]. In spite of this benefit, recent IoMT studies often found to using traditional methods [13, 14, 15, 16, 17]. This reluctance to using DL models can probably be characterised by the resource required by deep models which seem to be overwhelming for resource constrained environment.

A solution to a given physiological bio-signal related task, offered by a DL method, can be decomposed to its components including a pre-processing, the model itself, and a post-processing, as shown in Figure 1. Since the pre-processing is mainly used to prepare an environment, remaining two components were considered as variable. This study decouples the CNN models from post-processing to clearly understand their relative importance, later their complexities were varied to leverage the strengths of their combinations in order to find a solution customised for a resource constrained environment. Considering a DL-based solution as a compositional optimisation of its components, that is suitable for resource constrained environment, were not actively sought in recent IoMT studies, which could be an attractive alternative to traditional approaches currently used.

A task of QRS-detection and R-peak localisation was selected as a case study to explore the strategy of incrementally increasing the complexity of a DL-based method’s components (DL model and post-processing) to offer an IoMT solution for continuous ECG signal monitoring. Monitoring physiological signals using wearable sensors is increasing in recent time and monitoring ECG signal is of particular importance since it can detect a variety of heart problems, including arrhythmias, coronary heart disease, heart attacks, and cardiomyopathy [19, 20, 21, 22]. ECG signal consists of P-wave, QRS-complex, and T-wave. The QRS-detection literature contains traditional digital filter-based approaches [23, 13, 14, 15, 16, 17] and classical machine-learning approaches [24, 25, 26] to detect QRS and localise R-peaks. Recent deep-learning-based QRS-detection and R-peak localisation studies reported superior performance across multiple validation-datasets, which involves the use of complex CNN-model-architectures and essential post-processing [5, 27, 28, 29, 30, 31]. The post-processing (PP) in these studies can be summarised as -

• •

  minimal PP (basic Salt and Pepper filter) [27],

• •

  moderate PP (use domain-knowledge to group 1s as QRS candidate, i.e. 64ms equivalent sample-length) [5, 30, 31, 29], and

• •

  advanced PP (use domain-knowledge to filter close QRS neighbours, i.e. minimum R-R distance to be 100 milliseconds) [27, 31, 5]

Sometime it is unclear if the resultant QRS-detection performance is due to the strength of the model or the post-processing and very limited information is available to quantify the post-processing effect. In addition, none of these deep learning based approaches consider limitations of resource constrained devices for their algorithm design.

This study identifies relative complexities of CNN models and post-processing so that a set of combinations can be obtained. CNN model’s complexity was increased by increasing its depth, since the depth parameter significantly contributes to a model’s learning capacity [2] and the complexity of post-processing was dealt by segregating to three components based on the amount of domain-knowledge used. A set of model-and-postprocessing configurations was then identified to be suitable for corresponding target environments’ resource capacity. To the best of our knowledge, the selection of a CNN model by incrementally increasing its complexity, based on a target resource-constraint environment, and leveraging the strength of post-processing to propose a set of configurations is unique of its kind.

The implementation is released as open source on GitHub.¹¹1https://github.com/deakin-deep-dreamer/qrs_postprocess012

An application framework that was shown in the Abstract, has been implemented in Raspberry Pi and shown in Figure 3.

In this Raspberry Pi 4 model-B based implementation, ECG signal was received from the sensor over a Bluetooth low-energy interface, processed by our proposed set of QRS detection algorithms (set of configurations combining CNN model and post-processing complexity), and finally, sends the ECG signal, along with R-peak annotations, to the cloud using web interface. Other low-resource devices were not considered for implementation in this study, rather a resource-configuration-based assumption has been discussed.

Figure 4 shows the variation in F1-scores with increasing depths (scaled using $log_{2}D$) of the model using the post-processing (PP) methods - minimal, moderate, and advanced (indexed as PP 1-3, shown in Algorithm 1, 2 3).

PP-1 F1-scores of 2-layer CNN (in Figure 4-a) are the lowest and increase through 4 and 8-layer until 16-layer. The F1-scores between 16 and 32-layer-depth marginally increases for INCART, QT, EDB, STDB, and SVDB, marginally decreases for the NSTDB, however, increases significantly for the TWADB (around 7%). Beyond 32-layer-depth, the F1-scores declined for all the datasets, where the INCART and TWADB decreased comparatively at a higher margin of around 3%. The NSTDB F1-score response varied around only 2% (between 86.5% and 89.8%) across the depths, similar to a group of datasets (QT, EDB, STDB, SVDB), which varied $<$2%, but the INCART and TWADB F1-scores varied around 8% (between 87% and 95%) and 14% (between 81% and 95%).

With the moderate and advanced PP (in Figure 4-b,c), the F1-score growth variation flattens better compared to the minimal PP (in Figure 4-a) across the datasets. The advanced post-processed F1-score response is almost similar to the moderate post-processed scores across network-depths with an exception of TWADB, that the advanced PP shows significant improvement starting from 2-layer-deep network (improved by around 13% margin, from 83% in moderate to 96% in advanced PP).

Figure 5 shows three post-processing steps’ relative importance across the validation datasets for networks with 2, 4, and 64 layer depths (intermediate depths not reported since the variation increases marginally, as well as, to keep it brief for better explainability).

Three post-processing methods (indexed as PP 1-3) are represented as blue, yellow, and green color-bar in order. In Figure 5-a, the PP-1 (blue-bar) shows the lowest F1-scores, compared to the other two PPs, while its lowest score is around 81% for TWADB. PP-2 (yellow-bar) is higher than PP-1 (blue-bar), for the 2-layer-depth (Figure 5-a), where this margin is $<$5% for all the datasets. The PP-3, in the same Figure 5-a, on the other hand, is superior to the PP-2 marginally for all, but the TWADB, for which the margin is around 15%. With 4-layer deep CNN (Figure 5-b), the scores of PP-1 improved proportionately, but the TWADB and NSTDB marginally improved ($<$2%). The highest 64-layer deep CNN (Figure 5-c) almost equalises the effects of the PP methods (PP 1-3) for the datasets, including the TWADB, where PP-1 is lower than PP-3 by only around 1% margin.

The characteristic curves, in the Figure 6, depicts the score variation of different PPs w.r.t. the network-depths, for each validation dataset. For almost all the datasets, the PP-2 and PP-3 are indistinguishable, except TWADB, for which all PPs tend to come closer at 32-layer-deep CNN (shown in Figure 6-e). The PP-2 is marginally ($<$1%) lower than PP-3 for datasets like QTDB, and STDB (in Figure 6-b, and d) across network depths, but this difference is in the range of 2-10% for datasets like INCART, EDB, NSTDB, (in Figure 6-a, c, f) and $>$12% for TWADB (in Figure 6-e).

The characteristic curves in Figure 7 shows the response of F1-scores for network-depths w.r.t. PPs, across the validation datasets. The shallow 2-layer CNN, with PP-1, yields the lowest F1-scores across the datasets, but with the PP-2, the score improves (with margin in the range around 1-6%). The PP-3, on the other hand, always achieves marginal F1-score improvement compared to the moderate post-processing, with the exception of TWADB (Figure 7-e), that improved with around 13% margin. Unlike the INCART and TWADB, the 8, 16, and 32-layer-deep networks are almost indistinguishable for the rest validation-datasets across PPs.

CNN models were trained with large amount of data in computing hardware (GPU server with Nvidia Tesla Volta V100-SXM2-32GB), where the amount of additional time, required for each training epoch for increasing depths of CNN models (shown in Table II), increases exponentially.

The table also shows the number of CNN model parameters, and memory requirement (in kB), which also grow exponentially with the increase of model depths. The models requires more memory, due to the number of parameters, for higher depths. The validation or test time, on the other hand, is constant across network depths, as shown for INCART dataset, run in Raspberry Pi 4. Each post-processing method scans the output prediction-stream sequentially and updates the individual bits of the stream as required, thus, their run-time complexity is $\mathcal{O}(n)$.

IoMT solutions often seek traditional filter-based algorithms for embedded device-based resource constrained environments and reluctant to use deep-learning-based algorithms due to their high resource requirement, in spite of their strengths including noise resistance, automated feature extraction and learning. In a sensor-based end-to-end solution, a shallow CNN model has the capacity to offer better performance by leveraging other core components, such as post-processing. To do that, in this study, the complexity of CNN models (only the depth parameter varied, since it solely dominates a model’s learning capacity) were incrementally increased and combined with a set of post-processing. A use case of IoMT based ECG signal monitoring was chosen, where the post-processing was segregated based on an increasing use of domain-knowledge including minimal PP-1 (basic salt & pepper filter), moderate PP-2 (minimum QRS extent 64 milli-seconds), and advanced PP-3 (minimum R-R distance to be 200 milli-seconds). Results of this study shows that it is possible to design an optimal DL solution with known target performance characteristics and resource (computing capacity and memory) constraints.

A combination of a shallow 2-layer CNN and PP-1 seems capable to achieve more than 85% F1-score for almost all the datasets (except TWADB, that scores around 81%). If this is considered as a basic configuration, a superior performance can be achieved by altering the post-processing only. For example, with PP-2, and PP-3, the score of a shallow 2-layer CNN improves by 3% and 15% w.r.t. PP-1. Different CNN model architecture and complexity were used in the QRS detection literature [5, 28, 43, 27], however, the use of a shallow 2-layer CNN model is scarce, where the emphasis was put on post-processing. The strength of post-processing can be leveraged to process a primitive CNN model’s output to achieve comparable performance across a broad range of datasets. Such a configuration would be embedded device friendly, both in terms of CNN model (2-layer CNN has the lowest memory footprint), and post-processing (linear run-time complexity).

The quality of CNN model’s prediction-stream improves with the increase of its complexity. This can be understood by observing the improvement of PP-1 F1-scores for CNNs with increasing depths (characteristic curve, shown in Figure 6). The PP-1 is a basic Salt & Pepper filter, which was found to reach almost the highest level scores of PP-3 for CNN model of depth-8 (QT, and STDB, in Figure 6-b,d), depth-16 (INCART, EDB, and NSTDB, in Figure 6-a,c,f), and depth-32 (TWADB, in Figure 6-e). Deep CNN models (depth $>$8 layers) learn better from the training data [44], yield better prediction-stream and reduces the effect of post-processing, although, each dataset has its own depth requirement to achieve the top score. It is particularly important to observe that a shallow 2-layer CNN model may be difficult to train to learn a complex function [45, 44] (i.e. if an input sample belongs to a QRS region or not), that requires PP-2 (for most datasets, in Figure 6,7) or PP-3 (for all datasets) to perform better than PP-1, because it seems that the use of domain-knowledge-based post-processing is complementing the shortcoming of the capacity of a shallow model. The selection of PP-2 or PP-3 (instead of PP-1) with deep CNN models (depth $>$8 layers) was found to be a better choice to achieve the top performance for most of the datasets (except TWADB, for which PP-3 is required, instead of PP-2). This requirement, however, may not comply with a target resource-constrained device, which has memory or computation limitation, thus, requires a shallow CNN model.

For a CNN model to have greater generalisability, it is required to perform almost at the same level for a broad range of test ECG data. The characteristic curves of datasets, w.r.t. PPs, or CNN depths (in Figure 6, 7), show the peculiarity of some of them (i.e. INCART, and TWADB), which achieves better performance with a resource-constrained configuration (shallow 2-layer CNN with PP-3), but requires a deep CNN model (8 or 16-layer CNN with PP-3) to achieve the peak performance. A deep CNN model seems trying to understand subtle patterns which enables it to produce better prediction-stream for a wide range of test records (with greater inter and intra-patient variance). As one can assume, the minimum configuration (shallow 2-layer CNN with PP-3) may not guarantee a similar level of performance for a test ECG record if it is much different than the training set and the shallow CNN likely be failing to learn the general features during training. Data diversity is one of the challenges in designing a universal QRS-detector [46], and there are studies in the literature, which cross-validates the model with fractions of a single dataset or a few datasets. Having a deep CNN model to generate better prediction-stream seems beneficial for a clinical context where the performance is the prime requirement.

A suitable configuration can be derived for a target resource-constraint device by matching its memory and computation capacity with those of the CNN models. These observations can be summarised as below points, considering a simplified assumption that the validation datasets represent a universal set of datasets -

• •

  Target device has extreme low memory and computing capacity: a combination of 2-layer CNN and PP-3 would yield the best performance.

• •

  Target device has low memory and computing capacity: a combination of 8-layer CNN and PP-3 yields the top performance (for all but the TWADB).

• •

  Target device has sufficient resources: a combination of 32-layer CNN and PP-2 or PP-3 yields the top performance for all the datasets.

Deep-learning models often are not a default choice of method in an IoMT end-to-end applications which require such models to deploy in resource constrained environment. Classical filter-based approaches are commonly encountered in such context due to their small memory and computation footprint. In this study, we explored ways for optimising DL solutions for improving their deployability in resource (computation capacity and memory) constrained devices. In a QRS-detection case study, it was shown that by effectively combining CNN models and post-processing by varying their complexities could facilitate to yielding a set of configurations suitable for a range of target resource constrained environments. It was found that a narrow 2-layer CNN can achieve comparable performance with a suitable post-processing, targeting a low-resource environment, however, $>$8-layer CNN would be required to achieve the top scores, which may be suitable for comparatively rich target environments. Results of this study pave ways to design DL solution for known resource constraint and target performance characteristics. Finally, this study contributes in improving deploy-ability of DL solutions in resource constrained devices.

CNN model’s complexity optimisation to yielding a compositional solution to test on real-life data for a given task (i.e. QRS-detection and R-peak localisation) to monitor performance and power-consumption of resource constrained environment could be a promising future direction.

This research was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government.