Real-Time Cardiac Cine MRI with Residual Convolutional Recurrent Neural Network

Real-time cardiac cine MRI (RT-cine), compared to retro-cine, requires neither ECG gating nor breath-holding, which can be applied to a more general patient cohort and simplify the scanning process (Yuan et al., 2000). To achieve a high temporal resolution ($<$50ms) and an acceptable spatial resolution ($<$2mm), highly undersampled data ($>$10x acceleration) needs to be collected for RT-cine, which imposes a significant challenge for image reconstruction. Compressed sensing (CS) based approaches have been proposed for dynamic cardiac MR (CMR) (Chen et al., 2014; Kido et al., 2016; Hansen et al., 2012; Kellman et al., 2009; Jung et al., 2009; Gamper et al., 2008) and have shown good reconstruction quality under high acceleration. The application of CS to RT-cine, however, is limited by the slow reconstruction speed due to iterative algorithms and tricky hyperparameter tuning. Deep learning based methods, on the other hand, can potentially achieve much faster reconstruction speed and thus have drawn many attentions in recent years (Schlemper et al., 2017; Hammernik et al., 2018; Han et al., 2019; Hauptmann et al., 2019; Jin et al., 2019). Qin et al. (Qin et al., 2018) developed a convolutional recurrent neural network for dynamic CMR. While these studies show promising results, there are several limitations: First, simulated undersampled data from retro-cine rather than real RT-cine data are used for evaluation. Second, the acceleration rates are usually lower than 10x. Third, algorithms are tested on synthesized single-coil data rather than multi-coil data. Fourth, the reconstruction quality evaluations lack clinical assessment.

In this study, we propose a residual convolutional recurrent neural network (Res-CRNN) for RT-cine MRI. Our deep learning model is evaluated on highly accelerated (12x) multi-coil RT-cine data and compared to CS. To the best of our knowledge, this is the first work applying the deep learning approach to real Cartesian RT-cine MRI and it is evaluated by radiologists.

Since it is almost impossible to obtain ground truth (i.e., fully sampled data) for RT-cine, we adopt the strategy of training the deep learning model on retro-cine data. Then the trained model is applied to the acquired undersampled multi-coil RT-cine data directly from scanners for image reconstruction.

Data collection: We collected retro-cine data of 51 patients (total 343 slices) with a bSSFP sequence on a clinical 3T scanner (uMR 790 United Imaging Healthcare, Shanghai, China) with the approval of local IRB. We also collected RT-cine data of 27 slices from two patients, which were acquired using a bSSFP sequence using variable Latin Hypercube undersampling (Lyu et al., 2019). Imaging parameters include: imaging matrix: 192 x 180, TR/TE = 2.8/1.3 ms, spatial resolution = $1.82\times 1.82$ $mm^{2}$, and temporal resolution = 34 ms and 42 ms for retro-cine and RT-cine, respectively. Both retro- and RT-cine data were acquired using phased-arrayed coils.

Res-CRNN: Figure 1 depicts the proposed Res-CRNN. The network includes: three bi-directional convolutional RNN layers to model dynamic information (Qin et al., 2018), the data consistency layer (Schlemper et al., 2017), as well as two levels of residual connections, which promote the network to learn high-frequency details. To reduce the GPU memory consumption and speed up reconstruction, one extra 2DConv layer is added within each bi-directional ConvRNN layer to reduce the number of feature maps in hidden states. All Conv kernels are $3\times 3$ and the filter numbers are k=48 for bi-directional ConvRNN layers and k=2 for other 2DConv layers.

Model training: For model training, the retro-cine data were retrospectively undersampled using the same 12x acceleration sampling mask as in RT-cine and then fed into the neural network as input. MSE and SSIM (Zhao et al., 2015) were used as the training loss. Images from each coil were reconstructed independently and then combined using the root sum of square method.

Evaluation: For comparison, the same RT-cine data was reconstructed by CS using BART (Tamir et al., 2016). Temporal total variation and spatial wavelets were adopted and ADMM was used for optimization with maximum iterations of 100. CS hyperparameters were heuristically optimized on one representative data. Coil sensitivity maps were calculated using ESPIRiT (Uecker et al., 2014) from temporally averaged data. Res-CRNN and CS were evaluated on a workstation (Intel Xeon E5-2630 CPU, 256G memory and Nvidia Tesla V100 GPU).

The reconstruction results were independently evaluated by four experienced cardiologists based on four radiologists (Figure 3), where a score in the 1-5 scale (1 as unacceptable and 5 as perfect) was assigned. The pair of reconstruction videos from CS and Res-CRNN were presented side-by-side in a randomized and blind fashion to radiologists. Statistical significance was calculated by repeated measures ANOVA.

Figure 2 shows examples of RT-cine reconstruction results. Res-CRNN achieves less noise and aliasing artifacts than CS. Figure 3 shows the evaluation from four radiologists. Res-CRNN reconstruction receives significantly better scores for noise and artifacts as well as motion, but lower scores for sharpness than CS (all P$<$0.001). There is no significant difference in diagnostic value between CS and Res-CRNN (P$=$0.48). No significant difference among the four radiologists was observed (all P$>$0.1). Reconstruction speed of Res-CRNN is more than ten times faster than CS (318.3s vs. 25.7s on average).

Unlike other studies using synthetic data from retro-cine (Qin et al., 2018; Schlemper et al., 2017; Sandino et al., 2019) or processed coil-combined data from RT-cine (Hauptmann et al., 2019), we proposed a deep learning model, Res-CRNN, for real multi-coil RT-cine MRI reconstruction. The deep learning method presents superior image quality over CS and is favored by the radiologists in noise/artifacts reduction and motion depiction. Moreover, the reconstruction speed of the deep learning method is significantly faster than CS, making it suitable for daily clinical usage. With implementation optimization, the reconstruction speed of Res-CRNN can be further improved.