distributed mri reconstruction using gadgetron-based cloud computing
Post on 11-Mar-2017
Embed Size (px)
Distributed MRI Reconstruction Using Gadgetron-BasedCloud Computing
Hui Xue,1* Souheil Inati,2 Thomas Sangild Srensen,3 Peter Kellman,4 andMichael S. Hansen1
Purpose: To expand the open source Gadgetron reconstructionframework to support distributed computing and to demonstratethat a multinode version of the Gadgetron can be used to provide
nonlinear reconstruction with clinically acceptable latency.Methods: The Gadgetron framework was extended with newsoftware components that enable an arbitrary number of
Gadgetron instances to collaborate on a reconstruction task.This cloud-enabled version of the Gadgetron was deployed onthree different distributed computing platforms ranging from a
heterogeneous collection of commodity computers to thecommercial Amazon Elastic Compute Cloud. The Gadgetron
cloud was used to provide nonlinear, compressed sensingreconstruction on a clinical scanner with low reconstructionlatency (eg, cardiac and neuroimaging applications).
Results: The proposed setup was able to handle acquisitionand 11-SPIRiT reconstruction of nine high temporal resolution
real-time, cardiac short axis cine acquisitions, covering theventricles for functional evaluation, in under 1 min. A three-dimensional high-resolution brain acquisition with 1 mm3 iso-
tropic pixel size was acquired and reconstructed with nonlin-ear reconstruction in less than 5 min.
Conclusion: A distributed computing enabled Gadgetron pro-vides a scalable way to improve reconstruction performanceusing commodity cluster computing. Nonlinear, compressed
sensing reconstruction can be deployed clinically with lowimage reconstruction latency. Magn Reson Med 000:000000, 2014. VC 2014 Wiley Periodicals, Inc.
Key words: Gadgetron; distributed computing; nonlinear MRIreconstruction; open-source software
Image reconstruction algorithms are an essential part ofmodern medical imaging devices. The complexity of thereconstruction software is increasing due to the compet-
ing demands of improved image quality and shortened
acquisition time. In the field of MRI, in particular, the
image reconstruction has advanced well beyond simple
fast Fourier transforms to include parallel imaging (15),
nonlinear reconstruction (6,7), and real-time reconstruc-
tion (8,9). The increased interest in three-dimensional
(3D) acquisitions and non-Cartesian sampling has
boosted the computational demands further. For appli-
cations where lengthy acquisition and reconstruction
time is prohibited by physiological motion, biological
processes, or limited patient cooperation, the reconstruc-
tion system is under further pressure to deliver images
with low latency in order to keep up with the ongoing
study.While the need for fast image reconstruction is grow-
ing, most published reconstruction algorithms, especiallythose relying on iterative solverssuch as image domaincompressive sensing (6,1012) and k-space SPIRiT andits regularized versions (7,13,14)do not come with theefficient reference implementations that would enableclinical use. In many cases, the algorithms are not imple-mented for online use on clinical scanners. Even if thedevelopers would like to integrate their reconstructionalgorithms for online use, the vendor-provided hardwareand software platform may have inadequate specifica-tions for a demanding reconstruction, or the availableprogramming window may be unsuited for integration ofnew reconstruction schemes. Consequently, there is agap between the number of new algorithms being devel-oped and published and the clinical testing and valida-tion of these algorithms. Undoubtedly, this is having animpact on the clinical adoption of novel nonlinear recon-struction approaches (eg, compressed sensing).
We have previously introduced an open-source plat-form for medical imaging reconstruction algorithmscalled the Gadgetron (15), which aims to partiallyaddress the aforementioned concerns. This platform isfreely available to the research community and industrypartners. It is platform independent and flexible for bothprototyping and commercial development. Moreover,interfaces to several commercial MR scanners have beendeveloped and are being shared in the research commu-nity. This simplifies the online integration of new recon-struction algorithms significantly, and the newalgorithms in research papers can be tested in clinicalsettings with less implementation effort. As a result,some groups have used Gadgetron for online implemen-tation and evaluation of their reconstruction methods(1618). Since the publication of the first version of theGadgetron, the framework has adopted a vendor-independent raw data format, the ISMRM Raw Data
1Magnetic Resonance Technology Program, National Heart, Lung andBlood Institute, National Institutes of Health, Bethesda, Maryland, USA.2National Institute of Mental Health, National Institutes of Health, Bethesda,Maryland, USA.3Department of Computer Science and Department of Clinical Medicine,Aarhus University, Aarhus, Denmark.4Medical Signal and Image Processing Program, National Heart, Lung andBlood Institute, National Institutes of Health, Bethesda, Maryland, USA.
Grant sponsor: National Heart, Lung and Blood Institute IntramuralResearch Program.
*Correspondence to: Hui Xue, Magnetic Resonance Technology Program,National Heart, Lung and Blood Institute, National Institutes of Health, 10Center Drive, Bethesda, MD 20814. E-mail: firstname.lastname@example.org
Additional Supporting Information may be found in the online version ofthis article.
Received 11 December 2013; revised 28 January 2014; accepted 18February 2014
DOI 10.1002/mrm.25213Published online 00 Month 2014 in Wiley Online Library (wileyonlinelibrary.com).
Magnetic Resonance in Medicine 00:0000 (2014)
VC 2014 Wiley Periodicals, Inc. 1
(ISMRMRD) format (19). This further enables sharing ofreconstruction algorithms.
Although this concept of an open-source platform anda unified ISMRMRD format shows great potential, theoriginal Gadgetron framework did not support distrib-uted computing across multiple computational nodes.Although the Gadgetron was designed for high perform-ance (using multiple CPU cores or graphics processingunits [GPUs]), it was originally implemented to operatewithin a single node or process. Distributed computingwas not integral to the design. Because reconstructionalgorithms increase in complexity, they may need com-putational power that would not be economical toassemble in a single node. The same considerations haveled to the development of commodity computing clusterswherein a group of relatively modest computers areassembled to form a powerful computing cluster. Anexample of such a cluster system is the National Insti-tutes of Health (NIH) Biowulf Cluster (http://biowulf.nih.gov). Recently, commercial cloud-based services havealso offered the ability to configure such commoditycomputing clusters on demand and rent them by thehour. The Amazon Elastic Compute Cloud (EC2) is anexample of such a service (http://aws.amazon.com/ec2).
In this paper, we propose to extend the Gadgetronframework to enable cloud computing on multiplenodes. With this extension, named Gadgetron Plus (GT-Plus), any number of Gadgetron processes can be startedat multiple computers (referred to as nodes), and adedicated interprocess controlling scheme has beenimplemented to coordinate the Gadgetron processes torun on multiple nodes. A large MRI reconstruction taskcan be split and run in parallel across these nodes. Thisextension to distributed computing maintains the origi-nal advantages of the Gadgetron framework. It is freelyavailable and remains platform-independent. As demon-strated in this paper, the nodes can even run differentoperating systems (eg, Windows or different distributionsof Linux) and have different hardware configurations.
The implemented architecture allows the user to setup a GT-Plus cloud in several different ways. Specifi-cally, it does not require a dedicated professional cloudcomputing platform. The GT-Plus cloud can be deployedon setups ranging from an arbitrary collection of net-worked computers in a laboratory (we refer to this as acasual cloud) to high-end commercial cloud systems,as demonstrated herein. In this paper, we demonstratethe GT-Plus cloud setup in three different scenarios anddemonstrate its flexibility to cope with variable computa-tional environments. The first cloud setup is a casualcloud consisting of seven personal computers situated inour laboratory at the National Institutes of Health. Thesecond setup uses NIHs Biowulf Cluster (20). The lastconfiguration is deployed on the commercial AmazonElastic Compute Cloud (Amazon EC2) (21).
To demonstrate the benefits of this extension, we usedthe cloud-enabled version of the Gadgetron to implementnonlinear 11-SPIRiT (7) for two-dimensional (2D) time-resolved (2Dt) and 3D imaging applications. We dem-onstrate that cardiac cine imaging with nine slices cover-ing the entire left ventricle (32 channels; acquiredtemporal resolution50 ms; matrix size 192 100;
acceleration factor5, 1.5 s per slice) can be recon-structed with 11-SPIRiT with a reconstruction time(latency
6. The client sends every piece of readout data to theGadgetron in the ISMRMRD format. These data aredeserialized by the Reader and passed through theGadget chain.
7. The reconstruction results are serialized by theWriter and sent back to the client via the socketconnection.
8. When the end of acquisition is reached, the Gadget-ron closes down the Gadget chain and closes theconnection when the last reconstruction result hasbeen passed back to the client.
GT-Plus: Distributed Computing Extension of Gadgetron