Data Consistent Deep Rigid MRI Motion Correction

Subject motion frequently corrupts brain magnetic resonance imaging (MRI) and obfuscates anatomical interpretation [Andre et al. (2015)]. Prospective motion correction strategies adapt the acquisition in real-time to adjust for measured rigid-body motion [Tisdall et al. (2012), Frost et al. (2019)]. Unfortunately, prospective strategies require altering clinical workflows, prolonging scantime, or interfering with standard acquisition parameters. Retrospective strategies correct motion algorithmically after acquisition, with or without additional motion measurements [Batchelor et al. (2005), Bammer et al. (2007), Polak et al. (2022)]. Retrospective motion correction without additional motion information is particularly appealing, because it does not require external hardware or pulse sequence modifications and enables retroactive correction of large, previously collected k-space datasets. This work develops a deep learning method for retrospective rigid motion correction. We focus on intra-slice motion correction in multi-shot acquisitions, which acquire multiple k-space segments per slice and are a workhorse of clinical screening exams [Mehan et al. (2014)].

Retrospective intra-slice motion correction is often formulated as joint optimization of rigid motion parameters and the underlying image [Haskell et al. (2019), Cordero-Grande et al. (2016)]. The result is a highly non-convex, ill-posed optimization problem with undesirable local minima. The challenging nature of joint estimation has inspired several deep learning alternatives that train standard supervised convolutional neural networks (CNNs) to reconstruct motion-free images directly from motion-corrupted inputs [Pawar et al. (2018), Duffy et al. (2021), Levac et al. (2022b), Singh et al. (2022)]. Alternatively, GANs can be used to provide priors on the reconstructed images [Küstner et al. (2019), Johnson and Drangova (2019)]. None of these methods incorporate the MR forward model or allow the reconstruction strategy to depend on the motion parameters. Further, while these approaches provide visually appealing reconstructions, they do not enforce data consistency between the estimated output and the acquired measurements. As a result, they are susceptible to producing hallucinations, i.e., visually plausible features inconsistent with the acquired measurements.

Our work is closely related to methods that combine neural networks with enforced data consistency. Haskell et al. (2019) use an iterative procedure where a CNN produces an initial motion-corrected image, which initializes an alternating optimization over the motion parameters and underlying image. Levac et al. (2022a) use a score-based generative model that estimates the log probability of a reconstruction. At test-time, motion parameters and the underlying image are optimized to find a solution that is both data consistent and has a high prior probability under the generative model. Both methods incorporate optimizations of the underlying image and motion parameters. Our proposed strategy reduces this joint optimization to an optimization over just the motion parameters.

Our key insight is that the joint optimization can be simplified using a deep neural network to learn a mapping from proposed motion parameters and corrupted k-space to a motion-corrected reconstruction. Then, inference simplifies to a test-time search solely over the motion parameters, which in turn imply a reconstructed image. This strategy eliminates the search over images while retaining the benefits of iterative, optimization-based motion parameter estimation. In particular, the discrepancy between acquired measurements and the estimated image and motion parameters can be automatically monitored during the optimization to reject failure cases with a poor quality reconstruction.

We demonstrate our intra-slice motion correction method on 2D fast spin echo (FSE) brain MRI. Our method produces reconstructions that are consistent with the acquired k-space measurements in the presence of inter-shot motion or provides an indicator when they are not. The method is trained on simulated pairs of corrupted and motion-free examples and generalizes to a proof-of-concept acquired k-space example. In both simulated and realistic data, our method achieves consistently high quality reconstruction and motion parameter estimation consistent with the forward imaging model.

There are two stages to our motion correction procedure: (1) training a network that maps motion parameters to reconstructions and (2) performing a test-time optimization that solves for the motion parameters for an unseen k-space example, which then maps to the final reconstruction via the trained network (Fig. 1). We first describe the MRI forward model and then detail its use in neural network training and motion parameter estimation.

Unlike standard supervised deep learning methods which do not incorporate information about the motion parameters into the structure of the neural network Pawar et al. (2018); Duffy et al. (2021); Levac et al. (2022b); Singh et al. (2022); Küstner et al. (2019); Johnson and Drangova (2019), our method uses a motion-dependent neural network. In this paper, we instantiate the reconstruction network $f(y,m;\theta)$ with a hypernetwork $h(\cdot;\theta_{h})$ Ha et al. (2016); Hoopes et al. (2021) operating on motion parameters $m$ to generate weights $\theta_{g}$ of a reconstruction subnetwork $g$:

The weights $\theta_{g}$ of the reconstruction subnetwork are never trained directly. The only trainable parameters in $f$ are $\theta_{h}$, the weights of the network $h$ that produces $\theta_{g}$. This hypernetwork architecture enables flexible prediction of a reconstruction subnetwork $g(\cdot;h(m;\theta_{h}))$ specific to the motion parameters, just as $A(m)$ is a function of the motion parameters.

The inputs to our hypernetwork $h$ are the motion parameters $m$ comprising 18 scalars representing x- and y- translations and a rotation for each of 6 k-space shots. $h$ consists of 6 fully connected layers with 256 units followed by convolutions to produce the reconstruction subnetwork weights $\theta_{g}$; $h$ contains 2.7M trainable parameters.

The input to the reconstruction subnetwork is the Autocalibrating Reconstruction for Cartesian imaging (ARC) reconstruction Brau et al. (2008) of the acquired signals, expressed as 88-channel real and imaginary components of k-space data from 44 receive coils (4 neck channels were discarded). This input is processed via 6 convolution layers in both image and frequency space Singh et al. (2022), where each 3$\times$3 convolution outputs 32 channel features. The reconstruction subnetwork $g$ outputs 88 k-space channels. We reconstruct the final image via the inverse Fourier transform and root-sum-of-squares coil combination.

We note that several other architectures are possible for $f(y,m;\theta)$, e.g., an architecture that takes motion parameters as additional network inputs. Identifying the optimal architecture for motion-parameterized neural networks is an important avenue for future research. Here, we use hypernetworks as one example formulation of $f(y,m;\theta)$ to demonstrate our general approach of test-time motion parameter estimation to produce a reconstructed image consistent with the acquired data.

Fig. 3 shows reconstructions on a simulated example. Our method HN produces sharper, more accurate reconstructions than ‘motion-naive’ ARC and Conv that do not incorporate motion information. It also provides reconstructions of similar quality to ‘motion-aware’ Model-Based-GT and HN-GT that have access to ground truth motion parameters.

Fig. 3a shows that our method outperforms motion-naive methods across the test set and is on par with the motion-aware methods. This result holds when other image quality metrics are used (Appendix B, Fig. 6). Fig. 3b shows that our method yields better reconstructions than motion-naive baselines for the vast majority of subjects. Fig. 3c demonstrates that our method recovers accurate motion parameters for the first four high-energy shots, with most inaccurate cases automatically rejected.

Our method does not reliably recover motion parameters for the last two shots. These low-energy shots play a relatively small role in the data consistency optimization but also have low impact on the resulting reconstruction. Of fifteen automatically rejected cases, motion occurred between the second and third shots in nine examples. This splits the first four high-energy shots between two positions and causes hard-to-correct artifacts.

Finally, Fig. 4 visually illustrates reconstruction for the realistic test example that combines acquired k-space data from the same subject in two different positions. HN provides the best reconstruction among methods that do not require access to the true motion parameters, demonstrating that it successfully handles real motion-corrupted signals despite using only simulated data during training.

This work develops a general deep rigid motion correction approach for multi-shot MRI that we evaluate on 2D FSE T2 FLAIR data. The proposed framework first learns a mapping between corrupted k-space data, true motion parameters, and high-quality reconstructions. At test-time, only the motion parameter estimates are optimized, yielding data-consistent reconstructions, thus alleviating the complexity of a joint search over images and motion parameters. The approach includes a strategy for automatically rejecting samples where the optimization fails. Our method reliably produces high-quality reconstructions in simulation and generalizes to a proof-of-concept realistic acquired k-space example.

While we demonstrate that our approach generalizes to acquired k-space data, our evaluation focuses on simulated data. We do not model through-plane motion, a known source of significant artifact. Less well-studied effects including intra-shot motion, spin history, and signal decay during the FSE echo train may also affect our method.

Future work will thus investigate, model, and correct these additional motion effects while retaining data-consistent reconstructions in larger scale, clinical k-space datasets. Beyond MRI, our strategy for learning physically consistent reconstructions given a partially-unknown forward model could be applied to other semi-blind inverse problems, e.g. in seismic imaging or computational photography.

Research reported in this paper was supported in part by GE Healthcare and by computational hardware provided by the Massachusetts Life Sciences Center. We also thank Steve Cauley for helpful discussions. Additional support was provided by NIH NIBIB (5T32EB1680, P41EB015896, 1R01EB023281, R01EB006758, R21EB018907, R01EB019956, R01EB017337, P41EB03000, R21EB029641, 1R01EB032708), NIBIB NAC (P41EB015902), NICHD (U01HD087211, K99 HD101553), NIA (1R56AG064027, 1R01AG064027, 5R01AG008122, R01AG016495, 1R01AG070988, RF1AG068261), NIMH (R01 MH123195, R01 MH121885, 1RF1MH123195), NINDS (R01NS0525851, R21NS072652, R01NS070963, R01NS083534, 5U01NS086625, 5U24NS10059103, R01NS105820), the Blueprint for Neuroscience Research (5U01-MH093765), part of the multi-institutional Human Connectome Project, the BRAIN Initiative Cell Census Network grant U01MH117023, and a Google PhD Fellowship.