Please note, this post is based on a preprint and the original paper has not yet been published after peer review.
Title: VORTEX: Physics-Driven Data Augmentations Using Consistency Training for Robust Accelerated MRI Reconstruction
Author(s) and Year: Arjun D. Desai*, Beliz Gunel*, Batu M. Ozturkler, Harris Beg† , Shreyas Vasanawala, Brian A. Hargreaves, Christopher Re, John M. Pauly, Akshay S. Chaudhari
Journal: Under-review for MIDL 2022
Can you imagine lying really, really still for at least 15 minutes? That is the reality of patients who need to complete a magnetic resonance imaging (MRI) scan. Even if you could keep still for that long, a scan could take 15 to 90 minutes! Patients need to lie as still as possible so that the MRI machine can capture images used to detect and diagnose diseases. Even the tiniest patient movement can distort the final image that is returned.
How do we maintain reconstruction quality in spite of the noise generated from patient movement and physiology and hardware difference between MRI machines? Noise in images distorts what would otherwise be the perfect image. Reducing time taken for image acquisition benefits both patients and clinicians, with the caveat that the final reconstructed image must be of the best possible quality sufficient for diagnosis. The UK performed approximately 300,000 MRI scans in 2019, which means that between 50 and 300 days (at a rate of 15-90 minutes per scan) each year is spent performing these scans. With a limited number of MRI machines available, there is a real need to ensure that they provide maximum utility. [1, 2]
One image acquisition process currently used is compressed sensing. Instead of recording the full sized raw image (imagine requiring the highest possible resolution for that photoshoot), a subset of the images’ points is used to reconstruct and generate the final image. This technique aims to use significantly fewer measurements to reconstruct the image than what was previously thought to be required. In recent years, deep learning techniques have been the model of choice for faster and more accurate image reconstruction. Briefly, the success of deep learning models in the medical image domain can be attributed to improved hardware capabilities and model architecture over time and availability of data.
A common challenge faced in medical imaging problems is the ‘small dataset’ problem – the amount of data with clinically verified labels (e.g. mapping a specific image to a type of condition) is scarce. In an ideal scenario, we would have an abundant supply of reference images to compare the reconstructed image against – the closer the generated image is to the reference, the better the model is at reproducing high-quality images. However, these reference images are often not readily available. Thus, the second challenge: how do we overcome the ‘small dataset’ problem while maintaining high-quality reconstruction techniques?
VORTEX
VORTEX is the authors’ proposed solution which maintains high-quality reconstructed images balancing the small dataset problem. VORTEX also considers a range of difficulty levels of images that need to be interpreted, since interpretation is dependent on both clinical diagnosis and image quality, to further mimic real-world clinical conditions.
Maintaining High-Quality Reconstructions
In the engineering world, image augmentation is a commonly used technique to overcome the ‘small dataset’ problem. Augmentation is the manipulation of the original image through a series of transformations. For example, the image can be rotated or shifted horizontally and vertically, returning a ‘new’ image. Instead of only having the original image, now the user has a few different copies of the original image, increasing the size of the dataset. However, the process of using only image augmentation techniques is not robust against clinically relevant noise in MRI images. Therefore, the authors combine both image augmentation and physics-driven methods to improve the model’s ability to reconstruct high quality images.
Physics-driven augmentation aims to model two different physical phenomena that occur: scanner-induced noise and patient-induced motion artifacts. To model noise, authors sample a noise value σ between a range of values chosen based on visually inspecting noise from clinical scans. Motion is more complex because it also has an added dimension of direction (e.g. moving from left to right, up and down). Briefly, the authors use random phase shifts in the frequency space to model rigid motion. A phase shift in the frequency domain is equivalent to translation in the time domain and by extension the image space.
Simulating Learning Curves
Not all images will have the same level of difficulty when it requires interpretation. The authors add an extra component known as curriculum learning. This technique helps the model learn over time by gradually making the tasks more difficult, like that of a student whose knowledge accumulates throughout their years of schooling. In this case, the more noise applied to the image, the more difficult it is for the model to reconstruct the image accurately.
Efficient Labeling
To overcome the ‘small dataset’ problem, the authors use a combination of labeled and unlabelled data to train their model. For the labeled data, they use the full raw image to train the model to reconstruct the image. In contrast, their dataset is unlabelled raw images which are first augmented (using both image and physics driven augmentations) before being used for reconstruction.
The Conclusion
Armed with augmentation techniques, a curriculum learning approach, and efficient labeling, the authors show that VORTEX is able to produce reconstructed images which are robust against clinically relevant noise. One of the authors’ findings was that including physics-based augmentations is important for maintaining robustness and consistency of reconstructed images, especially when the images are more challenging to interpret. Previously only including image augmentation methods to overcome the ‘small dataset’ problem did not translate well to domains which had little labeled data. VORTEX also showed better performance than only using fully labeled data for reconstruction of images.
Is it possible to simulate more complex types of motion, such as rotation? Are there other ways of undersampling the image such that the process could become even more computationally efficient and less reliant on labels? As the authors have written, there are plans to extend this work as VORTEX provides a stepping stone towards “building robust deep-learning based MR reconstruction models that can be safely deployed in clinics”.
[1] https://www.rcr.ac.uk/posts/nhs-must-do-more-future-proof-its-mri-capacity-say-imaging-experts [2] https://www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2020/01/Provisional-Monthly-Diagnostic-Imaging-Dataset-Statistics-2020-01-23.pdf