unsupervised deep basis pursuit - icsi | icsistellayu/publication/doc/2019basisismrms… ·...

Tamir et al., Unsupervised Deep Basis PursuitAbstract #0660 1

Unsupervised Deep Basis Pursuit:Learning Reconstructions without Ground-Truth Data

Jonathan I Tamir1, Stella X Yu1,2, and Michael Lustig1

1Electrical Engineering and Computer Sciences, University of California, Berkeley2International Computer Science Institute, University of California, Berkeley

Grant Support:NIH grants R01EB009690, R01EB019241, P41RR09784; Sloan Research Fellowship; Bakar Fellowship; GE Healthcare (research support)

www.jtsense.com


Speaker Name: Jonathan I. Tamir

I have the following financial interest or relationship(s) to disclose with regard to the subject matter of this presentation:

• Grant/research support: GE Healthcare• Employment: Subtle Medical• Ownership Interest: Subtle Medical

Declaration ofFinancial Interests or Relationships


Motivation

[1] S Diamond, arXiv, 2017. [2] J Schlemper, IPMI, 2017. [3] K Hammernik, MRM 2017. [4] J Zhang, CVPR 2018. [5] Aggarwal, IEEE TMI 2019.

Deep learning + iterative optimization = Best of Both Worlds

minx

1

2||y �Ax||2 +Q(x)

<latexit sha1_base64="IGuTpTslxcR91FNovP3cnP9UT1w=">AAACPHicdZBLSwMxFIUzvq2vqks3F4ugiGWmVqo7HxuXLVoVOmXIpBkNZjJDkhHLdH6YG3+EO1duXCji1rXpS1T0QuDjnJvk3uPHnClt24/WyOjY+MTk1HRuZnZufiG/uHSmokQSWicRj+SFjxXlTNC6ZprTi1hSHPqcnvvXR13//IZKxSJxqtsxbYb4UrCAEayN5OVP3JAJL3VDrK/8AG6zHLiBxCR1srSUQaczdNqwBQNOD7Ih3WadjleCTaitfz2x4eULdnFvb7u8W4E+7AyhUganaPeqgAZV9fIPbisiSUiFJhwr1XDsWDdTLDUjnGY5N1E0xuQaX9KGQYFDqpppb/kM1ozSgiCS5ggNPfX7jRSHSrVD33R2J1S/va74l9dIdLDbTJmIE00F6X8UJBx0BN0kocUkJZq3DWAimZkVyBU22WmTd86EMNwU/oezUtHZLpZq5cL+4SCOKbSCVtE6clAF7aNjVEV1RNAdekIv6NW6t56tN+u93zpiDe4sox9lfXwCowiuXQ==</latexit>

MoDL1: CNN regularizationwith weights wQ(x) = �||x�Rw(x)||22

<latexit sha1_base64="hc48Y61/P8KZYWR0X1dYT4OEM2o=">AAACOnicbVDLTgIxFO3gC/GFunTTSExgIZkZTXRjQnTjEow8EsBJp3SgofNI21HIMN/lxq9w58KNC41x6wdYYIwK3qTJ6Tn35N577IBRIXX9SUstLC4tr6RXM2vrG5tb2e2dmvBDjkkV+8znDRsJwqhHqpJKRhoBJ8i1Ganb/YuxXr8lXFDfu5bDgLRd1PWoQzGSirKylUq+5SLZsx04KMAz2GLK20FwNIIJHw1ieDj9YMSiq9iKvpW7OM7/dBVGI8u8Ma1sTi/qk4LzwEhADiRVtrKPrY6PQ5d4EjMkRNPQA9mOEJcUMxJnWqEgAcJ91CVNBT3kEtGOJqfH8EAxHej4XD1Pwgn72xEhV4iha6vO8aJiVhuT/2nNUDqn7Yh6QSiJh6eDnJBB6cNxjrBDOcGSDRVAmFO1K8Q9xBGWKu2MCsGYPXke1MyicVQ0K8e50nkSRxrsgX2QBwY4ASVwCcqgCjC4B8/gFbxpD9qL9q59TFtTWuLZBX9K+/wCHUOtug==</latexit>


Motivation

y(1)<latexit sha1_base64="da0ShEmu1V7z6Y+pHZ7b41lC/v0=">AAAB+XicdVDLSsNAFJ3UV62vqEs3g0Wom5C0lba7ohuXFewD2lgm00k7dDIJM5NCCfkTNy4UceufuPNvnPQBKnpg4HDOvdwzx4sYlcq2P43cxubW9k5+t7C3f3B4ZB6fdGQYC0zaOGSh6HlIEkY5aSuqGOlFgqDAY6TrTW8yvzsjQtKQ36t5RNwAjTn1KUZKS0PTHARITTw/macPScm5TIdm0bYajUq1XoNLcrUmtSp0LHuBIlihNTQ/BqMQxwHhCjMkZd+xI+UmSCiKGUkLg1iSCOEpGpO+phwFRLrJInkKL7Qygn4o9OMKLtTvGwkKpJwHnp7McsrfXib+5fVj5dfdhPIoVoTj5SE/ZlCFMKsBjqggWLG5JggLqrNCPEECYaXLKugS1j+F/5NO2XIqVvmuWmxer+rIgzNwDkrAATXQBLegBdoAgxl4BM/gxUiMJ+PVeFuO5ozVzin4AeP9C+N6k9c=</latexit>

y(N)<latexit sha1_base64="UNHL2UKLFsDDg3rmeoek5H9mMmo=">AAAB+XicdVDLSsNAFJ34rPUVdelmsAh1E9KHtN0V3biSCvYBbSyT6aQdOpmEmUkhhPyJGxeKuPVP3Pk3TvoAFT0wcDjnXu6Z44aMSmXbn8ba+sbm1nZuJ7+7t39waB4dd2QQCUzaOGCB6LlIEkY5aSuqGOmFgiDfZaTrTq8zvzsjQtKA36s4JI6Pxpx6FCOlpaFpDnykJq6XxOlDUry9SIdmwbYajUq1XoMLcrkitSosWfYcBbBEa2h+DEYBjnzCFWZIyn7JDpWTIKEoZiTNDyJJQoSnaEz6mnLkE+kk8+QpPNfKCHqB0I8rOFe/byTIlzL2XT2Z5ZS/vUz8y+tHyqs7CeVhpAjHi0NexKAKYFYDHFFBsGKxJggLqrNCPEECYaXLyusSVj+F/5NO2SpVrPJdtdC8WtaRA6fgDBRBCdRAE9yAFmgDDGbgETyDFyMxnoxX420xumYsd07ADxjvXw+3k/Q=</latexit>

y(i)<latexit sha1_base64="w5AfClOYz1vpBCwm3aW8rQCwmSc=">AAAB+XicdVDLSsNAFJ3UV62vqEs3g0Wom5C0lba7ohuXFewD2lgm00k7dDIJM5NCCfkTNy4UceufuPNvnPQBKnpg4HDOvdwzx4sYlcq2P43cxubW9k5+t7C3f3B4ZB6fdGQYC0zaOGSh6HlIEkY5aSuqGOlFgqDAY6TrTW8yvzsjQtKQ36t5RNwAjTn1KUZKS0PTHARITTw/macPSYlepkOzaFuNRqVar8EluVqTWhU6lr1AEazQGpofg1GI44BwhRmSsu/YkXITJBTFjKSFQSxJhPAUjUlfU44CIt1kkTyFF1oZQT8U+nEFF+r3jQQFUs4DT09mOeVvLxP/8vqx8utuQnkUK8Lx8pAfM6hCmNUAR1QQrNhcE4QF1VkhniCBsNJlFXQJ65/C/0mnbDkVq3xXLTavV3XkwRk4ByXggBpoglvQAm2AwQw8gmfwYiTGk/FqvC1Hc8Zq5xT8gPH+BTjZlA8=</latexit>

Under-sampledk-space

x(i)<latexit sha1_base64="SpLYkGTCSqYujQJ5tR0W5Efr+Lg=">AAAB+XicdVDLSsNAFJ3UV62vqEs3g0Wom5C0lba7ohuXFewD2lgm00k7dDIJM5NiCfkTNy4UceufuPNvnLQVVPTAwOGce7lnjhcxKpVtfxi5tfWNza38dmFnd2//wDw86sgwFpi0cchC0fOQJIxy0lZUMdKLBEGBx0jXm15lfndGhKQhv1XziLgBGnPqU4yUloamOQiQmnh+cp/eJSV6ng7Nom01GpVqvQaX5OKL1KrQsewFimCF1tB8H4xCHAeEK8yQlH3HjpSbIKEoZiQtDGJJIoSnaEz6mnIUEOkmi+QpPNPKCPqh0I8ruFC/byQokHIeeHoyyyl/e5n4l9ePlV93E8qjWBGOl4f8mEEVwqwGOKKCYMXmmiAsqM4K8QQJhJUuq6BL+Pop/J90ypZTsco31WLzclVHHpyAU1ACDqiBJrgGLdAGGMzAA3gCz0ZiPBovxutyNGesdo7BDxhvnzdOlA4=</latexit>

x(1)<latexit sha1_base64="kuesUbd+SpCfkuzQH9EqWmariLY=">AAAB+XicdVDLSsNAFJ3UV62vqEs3g0Wom5C0lba7ohuXFewD2lgm00k7dDIJM5NiCfkTNy4UceufuPNvnLQVVPTAwOGce7lnjhcxKpVtfxi5tfWNza38dmFnd2//wDw86sgwFpi0cchC0fOQJIxy0lZUMdKLBEGBx0jXm15lfndGhKQhv1XziLgBGnPqU4yUloamOQiQmnh+cp/eJSXnPB2aRdtqNCrVeg0uycUXqVWhY9kLFMEKraH5PhiFOA4IV5ghKfuOHSk3QUJRzEhaGMSSRAhP0Zj0NeUoINJNFslTeKaVEfRDoR9XcKF+30hQIOU88PRkllP+9jLxL68fK7/uJpRHsSIcLw/5MYMqhFkNcEQFwYrNNUFYUJ0V4gkSCCtdVkGX8PVT+D/plC2nYpVvqsXm5aqOPDgBp6AEHFADTXANWqANMJiBB/AEno3EeDRejNflaM5Y7RyDHzDePgHh75PW</latexit>

x(N)<latexit sha1_base64="4fj10aJurUAtF7uBFeL2VXvMIvw=">AAAB+XicdVDLSgMxFM3UV62vUZdugkWom2GmrbTdFd24kgr2Ae1YMmmmDc1khiRTLEP/xI0LRdz6J+78GzN9gIoeCBzOuZd7cryIUals+9PIrK1vbG5lt3M7u3v7B+bhUUuGscCkiUMWio6HJGGUk6aiipFOJAgKPEba3vgq9dsTIiQN+Z2aRsQN0JBTn2KktNQ3zV6A1Mjzk4fZfVK4OZ/1zbxt1WqlcrUCF+RiRSpl6Fj2HHmwRKNvfvQGIY4DwhVmSMquY0fKTZBQFDMyy/ViSSKEx2hIuppyFBDpJvPkM3imlQH0Q6EfV3Cuft9IUCDlNPD0ZJpT/vZS8S+vGyu/6iaUR7EiHC8O+TGDKoRpDXBABcGKTTVBWFCdFeIREggrXVZOl7D6KfyftIqWU7KKt+V8/XJZRxacgFNQAA6ogDq4Bg3QBBhMwCN4Bi9GYjwZr8bbYjRjLHeOwQ8Y718OLJPz</latexit>

Ground-truth

CNN DC

Repeat N1 times

x̂(i)<latexit sha1_base64="CTZYb/daKZcQri5fsIw6Zbq9HWg=">AAAB/3icdVDLSsNAFJ34rPUVFdy4GSxC3YSkrbTdFd24rGAf0MQymU7aoZMHMxOxxCz8FTcuFHHrb7jzb5z0ASp6YOBwzr3cM8eNGBXSND+1peWV1bX13EZ+c2t7Z1ff22+LMOaYtHDIQt51kSCMBqQlqWSkG3GCfJeRjju+yPzOLeGChsG1nETE8dEwoB7FSCqprx/aIyQT20dy5HrJXZreJEV6mvb1gmnU6+VKrQpn5GxBqhVoGeYUBTBHs69/2IMQxz4JJGZIiJ5lRtJJEJcUM5Lm7ViQCOExGpKeogHyiXCSaf4UnihlAL2QqxdIOFW/byTIF2Liu2oyCyp+e5n4l9eLpVdzEhpEsSQBnh3yYgZlCLMy4IBygiWbKIIwpyorxCPEEZaqsrwqYfFT+D9plwyrbJSuKoXG+byOHDgCx6AILFAFDXAJmqAFMLgHj+AZvGgP2pP2qr3NRpe0+c4B+AHt/Qson5bb</latexit> L

⇣x̂(i),x(i)

⌘

<latexit sha1_base64="FjAM2lwHE/pd3icSd8KP15CcYpY=">AAACKHicbZDLSgMxFIYz9VbrrerSTbAILUiZqYLuLLpx4aKCvUCnlkyaaUMzF5Iz0jLM47jxVdyIKNKtT2LazkJbfwj8fOcccs7vhIIrMM2JkVlZXVvfyG7mtrZ3dvfy+wcNFUSSsjoNRCBbDlFMcJ/VgYNgrVAy4jmCNZ3hzbTefGJS8cB/gHHIOh7p+9zllIBG3fyV7REYUCLiu8QWzIWiPSAQz6jjxqMkeYyLvJSc4hTh0RzYkvcHUOrmC2bZnAkvGys1BZSq1s2/272ARh7zgQqiVNsyQ+jERAKngiU5O1IsJHRI+qytrU88pjrx7NAEn2jSw24g9fMBz+jviZh4So09R3dOt1WLtSn8r9aOwL3sxNwPI2A+nX/kRgJDgKep4R6XjIIYa0Oo5HpXTAdEEgo625wOwVo8edk0KmXrrFy5Py9Ur9M4sugIHaMistAFqqJbVEN1RNEzekUf6NN4Md6ML2Myb80Y6cwh+iPj+wftnKcS</latexit>

Update weights

[1] S Diamond, arXiv, 2017. [2] J Schlemper, IPMI, 2017. [3] K Hammernik, MRM 2017. [4] J Zhang, CVPR 2018. [5] HK Aggarwal, IEEE TMI 2019.

Loss metric

Deep learning + iterative optimization = Best of Both Worlds


Challenge




Under-sampledk-space Ground-truth

CNN DC

Repeat N1 times

x̂(i)<latexit sha1_base64="CTZYb/daKZcQri5fsIw6Zbq9HWg=">AAAB/3icdVDLSsNAFJ34rPUVFdy4GSxC3YSkrbTdFd24rGAf0MQymU7aoZMHMxOxxCz8FTcuFHHrb7jzb5z0ASp6YOBwzr3cM8eNGBXSND+1peWV1bX13EZ+c2t7Z1ff22+LMOaYtHDIQt51kSCMBqQlqWSkG3GCfJeRjju+yPzOLeGChsG1nETE8dEwoB7FSCqprx/aIyQT20dy5HrJXZreJEV6mvb1gmnU6+VKrQpn5GxBqhVoGeYUBTBHs69/2IMQxz4JJGZIiJ5lRtJJEJcUM5Lm7ViQCOExGpKeogHyiXCSaf4UnihlAL2QqxdIOFW/byTIF2Liu2oyCyp+e5n4l9eLpVdzEhpEsSQBnh3yYgZlCLMy4IBygiWbKIIwpyorxCPEEZaqsrwqYfFT+D9plwyrbJSuKoXG+byOHDgCx6AILFAFDXAJmqAFMLgHj+AZvGgP2pP2qr3NRpe0+c4B+AHt/Qson5bb</latexit>

Loss metric

???

What to do when there is no ground-truth data?

Stella Yu


Noise2Noise1

CNN with SURE2,3

Deep Image Prior4,5

Convolutional sparse coding6

……

ChallengeWhat to do when there is no ground-truth data?

[1] J Lehtinen, ICML 2018. [2] S Soltanayev, NIPS 2018. [3] A Metzler, arXiv 2018. [4] D Ulyanov, CVPR 2018. [5] D Van Veen, arXiv 2018. [6] F Ong, ISMRM 2018

minw

||y �AGw(z)||2<latexit sha1_base64="OTOC9RoH+RO6V6XpeR/NoGd/qm4=">AAACLnicdZBLS8NAEMc3vo2vqkcvi0WoB0taK7U3H4geK9gHtCVstpu6uNmE3Y2lpvlEXvwqehBUxKsfw22bgIoOLPz2PzPMzN8JGJXKsl6MqemZ2bn5hUVzaXlldS2zvlGXfigwqWGf+aLpIEkY5aSmqGKkGQiCPIeRhnNzOso3bomQ1OdXahCQjod6nLoUI6UlO3PW9ii3o7aH1LXjwn5sDofpZwD3YMLRcXxup9yPc2nJ3e5waBehncla+Uplv3RYhhM4SKFcgoW8NY4sSKJqZ57aXR+HHuEKMyRlq2AFqhMhoShmJDbboSQBwjeoR1oaOfKI7ETjc2O4o5UudH2hH1dwrH7viJAn5cBzdOVoTfk7NxL/yrVC5R52IsqDUBGOJ4PckEHlw5F3sEsFwYoNNCAsqN4V4mskEFbaYVObkF4K/4d6MV/YzxcvS9mjk8SOBbAFtkEOFEAZHIELUAU1gME9eASv4M14MJ6Nd+NjUjplJD2b4EcYn19Kb6lQ</latexit>

Generator networkrandom vector z


Goal

Learn network parameters directly from under-sampled data


Approach




Under-sampledk-space

CNN DC

Repeat N1 times

x̂(i)<latexit sha1_base64="CTZYb/daKZcQri5fsIw6Zbq9HWg=">AAAB/3icdVDLSsNAFJ34rPUVFdy4GSxC3YSkrbTdFd24rGAf0MQymU7aoZMHMxOxxCz8FTcuFHHrb7jzb5z0ASp6YOBwzr3cM8eNGBXSND+1peWV1bX13EZ+c2t7Z1ff22+LMOaYtHDIQt51kSCMBqQlqWSkG3GCfJeRjju+yPzOLeGChsG1nETE8dEwoB7FSCqprx/aIyQT20dy5HrJXZreJEV6mvb1gmnU6+VKrQpn5GxBqhVoGeYUBTBHs69/2IMQxz4JJGZIiJ5lRtJJEJcUM5Lm7ViQCOExGpKeogHyiXCSaf4UnihlAL2QqxdIOFW/byTIF2Liu2oyCyp+e5n4l9eLpVdzEhpEsSQBnh3yYgZlCLMy4IBygiWbKIIwpyorxCPEEZaqsrwqYfFT+D9plwyrbJSuKoXG+byOHDgCx6AILFAFDXAJmqAFMLgHj+AZvGgP2pP2qr3NRpe0+c4B+AHt/Qson5bb</latexit>

A(i)<latexit sha1_base64="qSjkWGiG7Y08Nc8I431wCyO4o5o=">AAAB+HicbVDLSsNAFL3xWeujUZduBotQNyWpgi6rblxWsA9oY5lMJ+3QySTMTIQa+iVuXCji1k9x5984abPQ1gMDh3Pu5Z45fsyZ0o7zba2srq1vbBa2its7u3sle/+gpaJEEtokEY9kx8eKciZoUzPNaSeWFIc+p21/fJP57UcqFYvEvZ7E1AvxULCAEayN1LdLvRDrkR+gq4e0wk6nfbvsVJ0Z0DJxc1KGHI2+/dUbRCQJqdCEY6W6rhNrL8VSM8LptNhLFI0xGeMh7RoqcEiVl86CT9GJUQYoiKR5QqOZ+nsjxaFSk9A3k1lMtehl4n9eN9HBpZcyESeaCjI/FCQc6QhlLaABk5RoPjEEE8lMVkRGWGKiTVdFU4K7+OVl0qpV3bNq7e68XL/O6yjAERxDBVy4gDrcQgOaQCCBZ3iFN+vJerHerY/56IqV7xzCH1ifP8kCkoQ=</latexit>

Loss metric

L⇣A(i)x̂(i),y(i)

⌘

<latexit sha1_base64="d4d/Rr+6F133NbQnBQET6Ha5JCs=">AAACOHicbZDLSgMxFIYzXmu9VV26CRahBSkzVdBl1Y0LwQr2Ap1aMmmmDc1cSM6IZZjHcuNjuBM3LhRx6xOYtiNo6w+Bn++cQ875nVBwBab5bMzNLywuLWdWsqtr6xubua3tugoiSVmNBiKQTYcoJrjPasBBsGYoGfEcwRrO4HxUb9wxqXjg38AwZG2P9HzuckpAo07uyvYI9CkR8WViC+ZCYQwcNz5NbuMCLyZ2n0D8A++TlB7gFOFh2iZ5rw/FTi5vlsyx8KyxUpNHqaqd3JPdDWjkMR+oIEq1LDOEdkwkcCpYkrUjxUJCB6THWtr6xGOqHY8PT/C+Jl3sBlI/H/CY/p6IiafU0HN052hbNV0bwf9qrQjck3bM/TAC5tPJR24kMAR4lCLucskoiKE2hEqud8W0TyShoLPO6hCs6ZNnTb1csg5L5eujfOUsjSODdtEeKiALHaMKukBVVEMUPaAX9IbejUfj1fgwPietc0Y6s4P+yPj6BjXIrco=</latexit>

Update weights

Impose loss directly on measured data1,2

[1] D Van Veen, arXiv 2018. [2] F Ong, ISMRM 2018



Approach




k-space

Loss metric

A(i)<latexit sha1_base64="qSjkWGiG7Y08Nc8I431wCyO4o5o=">AAAB+HicbVDLSsNAFL3xWeujUZduBotQNyWpgi6rblxWsA9oY5lMJ+3QySTMTIQa+iVuXCji1k9x5984abPQ1gMDh3Pu5Z45fsyZ0o7zba2srq1vbBa2its7u3sle/+gpaJEEtokEY9kx8eKciZoUzPNaSeWFIc+p21/fJP57UcqFYvEvZ7E1AvxULCAEayN1LdLvRDrkR+gq4e0wk6nfbvsVJ0Z0DJxc1KGHI2+/dUbRCQJqdCEY6W6rhNrL8VSM8LptNhLFI0xGeMh7RoqcEiVl86CT9GJUQYoiKR5QqOZ+nsjxaFSk9A3k1lMtehl4n9eN9HBpZcyESeaCjI/FCQc6QhlLaABk5RoPjEEE8lMVkRGWGKiTVdFU4K7+OVl0qpV3bNq7e68XL/O6yjAERxDBVy4gDrcQgOaQCCBZ3iFN+vJerHerY/56IqV7xzCH1ifP8kCkoQ=</latexit> L

⇣A(i)x̂(i),y(i)

⌘



Update weights

CNN DC

Repeat N1 times

y<latexit sha1_base64="ceqEUyhvEIiPXiXeyjDTF/IAWPA=">AAAB8HicdVDLSgMxFL1TX7W+qi7dBIvgaphpK627ohuXFexD2qFk0kwbmswMSUYYSr/CjQtF3Po57vwbM32Aih4IHM65l5x7/JgzpR3n08qtrW9sbuW3Czu7e/sHxcOjtooSSWiLRDySXR8ryllIW5ppTruxpFj4nHb8yXXmdx6oVCwK73QaU0/gUcgCRrA20n1fYD32A5QOiiXHvqw7lYsaWpDyilSryLWdOUqwRHNQ/OgPI5IIGmrCsVI914m1N8VSM8LprNBPFI0xmeAR7RkaYkGVN50HnqEzowxREEnzQo3m6veNKRZKpcI3k1lA9dvLxL+8XqKDujdlYZxoGpLFR0HCkY5Qdj0aMkmJ5qkhmEhmsiIyxhITbToqmBJWl6L/SbtsuxW7fFstNa6WdeThBE7hHFyoQQNuoAktICDgEZ7hxZLWk/VqvS1Gc9Zy5xh+wHr/AhCzkJg=</latexit>

x̂<latexit sha1_base64="YHgx7PBhMcwzyBTaAl8nSVBs2E4=">AAAB+XicdVDLSsNAFJ3UV62vqEs3g0VwFZK20nZXdOOygn1AE8pkOmmHTiZhZlIsIX/ixoUibv0Td/6Nkz5ARQ8MHM65l3vm+DGjUtn2p1HY2Nza3inulvb2Dw6PzOOTrowSgUkHRywSfR9JwignHUUVI/1YEBT6jPT86U3u92ZESBrxezWPiReiMacBxUhpaWia7gSp1A2RmvhB+pBlQ7NsW81mtdaowyW5WpN6DTqWvUAZrNAemh/uKMJJSLjCDEk5cOxYeSkSimJGspKbSBIjPEVjMtCUo5BIL10kz+CFVkYwiIR+XMGF+n0jRaGU89DXk3lE+dvLxb+8QaKChpdSHieKcLw8FCQMqgjmNcARFQQrNtcEYUF1VognSCCsdFklXcL6p/B/0q1YTtWq3NXKretVHUVwBs7BJXBAHbTALWiDDsBgBh7BM3gxUuPJeDXelqMFY7VzCn7AeP8C+yGUjw==</latexit>

⌘ fw(y)<latexit sha1_base64="hX04zI+TM/HqIzeBaxBLe2Z3Epc=">AAACCHicdZDLSsNAFIYn9VbrrerShYNFqJuQtJW2u6IblxXsBZoQJtNJO3RycWZSKSFLN76KGxeKuPUR3Pk2Tm+goj8MfPznHOac340YFdIwPrXMyura+kZ2M7e1vbO7l98/aIsw5pi0cMhC3nWRIIwGpCWpZKQbcYJ8l5GOO7qc1jtjwgUNgxs5iYjto0FAPYqRVJaTP7bIbUzH0HMsH8mh6yV3aXGJk/TMyRcMvV4vV2pVOIfzJVQr0NSNmQpgoaaT/7D6IY59EkjMkBA904iknSAuKWYkzVmxIBHCIzQgPYUB8omwk9khKTxVTh96IVcvkHDmfp9IkC/ExHdV53RF8bs2Nf+q9WLp1eyEBlEsSYDnH3kxgzKE01Rgn3KCJZsoQJhTtSvEQ8QRliq7nApheSn8H9ol3SzrpetKoXGxiCMLjsAJKAITVEEDXIEmaAEM7sEjeAYv2oP2pL1qb/PWjLaYOQQ/pL1/AeIpmo4=</latexit>



Approach




k-space

Loss metric


⇣A(i)x̂(i),y(i)

⌘



Update weights

minw

||y �Ax̂||2

subject to x̂ = fw(y)<latexit sha1_base64="8Sp6AavtYWVbed+n3IZy+EjkwUY=">AAACoHicdVFdT9swFHWyD6D7KuxxmmRRDbGHVWkpKjxMYuOFh00DaQW0uooc56b1cJzIdgZR6t/F/9jb/s2cthEb2q5k6ejec46vj6NccG2C4JfnP3j46PHa+kbrydNnz1+0N7fOdVYoBiOWiUxdRlSD4BJGhhsBl7kCmkYCLqKr43p+8QOU5pn8asocJimdSp5wRo1rhe1bEsGUy4oKPpUQ29YOJoWMnQJMRVJqZlGCr21FDNyYKuXS1pQdPJ83wxK/wytcfbBkRu90N3Y+D/uYkNp1YaCL6Dswg022tLlHx+9xEjZm13a3gaV92yIg47s1w3Yn6B4e7g0OhngJ9hswHOBeN1hUB63qNGz/JHHGihSkYYJqPe4FuZlUVBnOBNgWKTTklF3RKYwdlDQFPakWAVv8xnVinGTKHWnwovunoqKp1mUaOWa9sb4/q5v/mo0LkxxMKi7zwoBky4uSQrh8cP1bOObKxSVKByhT3O2K2Ywqyoz7oTqE5qX4/+C83+3tdftng87Rx1Uc6+gV2ka7qIeG6AidoFM0Qsx77R17n7zP/rZ/4n/xz5ZU31tpXqK/yv/2G7FsznA=</latexit>



Approach




k-space

Loss metric


⇣A(i)x̂(i),y(i)

⌘


Training on N data sets

Update weights

How to design f

minw

NX

i=1

||y(i) �A(i)x̂(i)||2

subject to x̂(i) = fw⇣y(i)

⌘

<latexit sha1_base64="IdHCLQUIV5D9ooOH9Ht4ivg8xKQ=">AAAC2HicdVFNb9NAEF2brxI+GuDIZUUESg9EcRqU9lCpwIVTVSTSVmRTa70eO0vXa8u7pok2ljiAEFd+Wm/8Cv4C68SGUsFIKz29eTNvdibIBFe63//huNeu37h5a+N2687de/c32w8eHqm0yBmMWSrS/CSgCgSXMNZcCzjJcqBJIOA4OHtd5Y8/Qq54Kt/pRQbThMaSR5xRbSm//ZMEEHNpqOCxhLBsPcOkkKGtAG1IQvUsiPB5aYiGuTYJl2UlsSJVJL7he155eoCXy1ppFuWp6fKtEj/HDfWypsiM/u5o5mXNLpf+ABNS2a4cVBF8AKaxTmufS1V43nTfw5HftDoviYBId6+OQHIez/RWi4AM/3zPb3f6vd3d7eHOCK/BiwaMhtjr9VfRQXUc+u0LEqasSEBqJqhSE6+f6amhueZMQNkihYKMsjMaw8RCSRNQU7M6TImfWibEUZrbJzVesZcrDE2UWiSBVVbTq6u5ivxXblLoaGdquMwKDZKtjaJC2LXh6so45LndolhYQFnO7ayYzWhOmbaXrZbQ/BT/HxwNet52b/B22Nl/Va9jAz1GT1AXeWiE9tEbdIjGiDljxzifnS/ue/eT+9X9tpa6Tl3zCP0V7vdfd57lzA==</latexit>


• Combine MoDL1 regularization with basis pursuit denoising2

Deep Basis Pursuit (DBP)

[1] HK Aggarwal, IEEE TMI 2019. [2] S Chen, Stanford technical report.

Known noise level

Nw(x) = x�Rw(x)<latexit sha1_base64="D0nvq/TSBDR7fuzljVvLuHpTXcg=">AAACPXicdZDJSwMxFMYzdat1q3r0EixCPThMF2l7EIpePEmVbtCWIZNm2tDMQpJRyzD/mBf/B2/evHhQxKtX080N+yDwy/fykfc+y2dUSMN41GILi0vLK/HVxNr6xuZWcnunLryAY1LDHvN400KCMOqSmqSSkabPCXIsRhrW4GzUb1wTLqjnVuXQJx0H9VxqU4ykksxkte0g2ceIhReROWbLDm+i9Axvo0N4Ar9v8Ah+Oa7mOcxkytBLpVy+WIATOJ5BIQ8zujGuFJhWxUw+tLseDhziSsyQEK2M4ctOiLikmJEo0Q4E8REeoB5pKXSRQ0QnHG8fwQOldKHtcXVcCcfqT0eIHCGGjqVejkYUf3sj8b9eK5B2sRNS1w8kcfHkIztgUHpwFCXsUk6wZEMFCHOqZoW4jzjCUgWeUCHMNoXzoZ7VMzk9e5lPlU+nccTBHtgHaZABBVAG56ACagCDO/AEXsCrdq89a2/a++RpTJt6dsGv0j4+Aeh8sOI=</latexit>

Estimates noise and aliasingminx

1

2||Nw(x)||22

subject to ||y �Ax||2 ✏<latexit sha1_base64="xvzUGYMQ2c8IWaWchyMHdqvjdCY=">AAACqnicdVFdb9MwFHXC1ygf6+CRF4sKGBJUSdap29uAF15AA9F2qC6V49x0Zo4TYgdauf5x/AXe+Dc4acqX4EqWju4959g+Ny4EVzoIvnv+pctXrl7bud65cfPW7d3u3p2xyquSwYjlIi/PYqpAcAkjzbWAs6IEmsUCJvHFi3o++Qyl4rl8p1cFzDK6kDzljGrXmne/khgWXBoq+EJCYjsPMalk4hSgDcmoPo9TvLSGaFhqk3Fpa4ojpSVlJrQmsut1w2NUmNd23mrMF7v/U/54vZ5HHyJMSG3fOKkq/ghMY51v/FoTx17hp3hr8mxpaykmAj5hAoXiIpcdAjL59eB5txf0j48PBkdDvAGHWzAc4LAfNNVDbZ3Ou99IkrMqA6mZoEpNw6DQM0NLzZkA2yGVgoKyC7qAqYOSZqBmpona4geuk+A0L92RGjfd3xWGZkqtstgx6z+ov2d181+zaaXTo5nhsqg0SLa5KK2ECwjXe8MJL11eYuUAZSV3b8XsnLolaLerOoTtT/H/wTjqhwf96M2gd/K8jWMH3UP30T4K0RCdoJfoFI0Q8x55r7yxN/Gf+G/99/50Q/W9VnMX/VF+8gMEk9Kv</latexit>

fw(y)<latexit sha1_base64="fBtb0k1i3IsKjixN6yQeeW1xUzQ=">AAACAXicdZDLSsNAFIYn9VbrrepGcDNYhLoJSWppl0U3LivYC7SlTKaTduhkEmYmSghx46u4caGIW9/CnW/jpBdQ0R8GPv5zDnPO74aMSmVZn0ZuZXVtfSO/Wdja3tndK+4ftGUQCUxaOGCB6LpIEkY5aSmqGOmGgiDfZaTjTi+zeueWCEkDfqPikAx8NObUoxgpbQ2LR96w7yM1cb3kLi0vMU7PhsWSZVbrVadSgzOoVK05OHYd2qY1Uwks1BwWP/qjAEc+4QozJGXPtkI1SJBQFDOSFvqRJCHCUzQmPY0c+UQOktkFKTzVzgh6gdCPKzhzv08kyJcy9l3dma0of9cy869aL1JefZBQHkaKcDz/yIsYVAHM4oAjKghWLNaAsKB6V4gnSCCsdGgFHcLyUvg/tB3TrpjO9XmpcbGIIw+OwQkoAxvUQANcgSZoAQzuwSN4Bi/Gg/FkvBpv89acsZg5BD9kvH8BaQOXhg==</latexit>


CNN step1

r = Rw(x)<latexit sha1_base64="ueyt2TuNbrV8wiOdhxZyJDRkV5U=">AAACF3icbZDLSsNAFIYn9VbrLerSzWAR6iYkVdCNUHTjsoq9QBvCZDpph04uzEzUEvIWbnwVNy4Ucas738ZJGkFbfxj4+M85zDm/GzEqpGl+aaWFxaXllfJqZW19Y3NL395pizDmmLRwyELedZEgjAakJalkpBtxgnyXkY47vsjqnVvCBQ2DGzmJiO2jYUA9ipFUlqMbfR/JketBDs9gzhix5Dp1Cj+5S2s/eJ8eOnrVNMxccB6sAqqgUNPRP/uDEMc+CSRmSIieZUbSThCXFDOSVvqxIBHCYzQkPYUB8omwk/yuFB4oZwC9kKsXSJi7vycS5Asx8V3Vma0oZmuZ+V+tF0vv1E5oEMWSBHj6kRczKEOYhQQHlBMs2UQBwpyqXSEeIY6wVFFWVAjW7Mnz0K4b1pFRvzquNs6LOMpgD+yDGrDACWiAS9AELYDBA3gCL+BVe9SetTftfdpa0oqZXfBH2sc3RBmgAw==</latexit>

DBP: Alternating minimization

[1] HK Aggarwall, IEEE TMI 2019. [2] Boyd, FTML 2011.

minx

1

2||x�Rw(x)||22

subject to ||y �Ax||2 ✏<latexit sha1_base64="NEFlmwg6Rftt6W/3sEirXWAvOg4=">AAACt3icdVFNc9MwEJVNgRI+GuDIRUMGphzI2G6YNLcClx4L07QdouCR5XUiKsuuJUMyiv4iB279N8hOUj4GdkYzT7tv30pvk1JwpYPg2vNv7dy+c3f3Xuf+g4eP9rqPn5ypoq4YjFkhiuoioQoElzDWXAu4KCugeSLgPLl839TPv0KleCFP9bKEaU5nkmecUe1Scfc7SWDGpaGCzySktvMSk1qmrgO0ITnV8yTDC2uIhoU2OZe2oThSVlFmQmsiu1pteGZh8WvcXhgV5qONt4Vvdv9G69VqFUefI0xIM6uVVXXyBZjGuliL3yji5VbQibxd2KYVEwFXmECpuChkh4BMf70+7vaC/mh0MDgc4jV4swXDAQ77QRs9tImTuPuDpAWrc5CaCarUJAxKPTW00pwJsB1SKygpu6QzmDgoaQ5qalrfLX7hMinOisodqXGb/b3D0FypZZ44ZvMH9XetSf6rNql1djg1XJa1BsnWg7JaOINws0Sc8sr5JZYOUFZx91bM5tRtRLvFNSZsf4r/D86ifnjQjz4MekfvNnbsomfoOdpHIRqiI3SMTtAYMW/gffKYl/ojP/Yzf76m+t6m5yn6I/yrn8vu1+A=</latexit>

Repeat N1 times

Data consistency step2

minx,r

1

2||x� r||22

subject to ||y �Ax||2 ✏<latexit sha1_base64="Amrb8qrrqGBdRLyQSFVA++xsHkU=">AAACrXicbVFNj9MwEHXC11K+Chy5WFQghKCbBCQ4LnDhWCS6u1B3K8eZZM06TrAd1Mr1v+MXcOPfYLeBwi4jWXqaeW888yZvBdcmSX5G8aXLV65e27s+uHHz1u07w7v3DnXTKQZT1ohGHedUg+ASpoYbAcetAlrnAo7ys3ehfvQNlOaN/GhWLcxrWkleckaNTy2G30kOFZeWCl5JKNzgMSadLLwCjCU1Nad5iZfP8G+onCUGlsbWXLrA9vxSUWZTZzO3Xvc8u3T4+U60Xi+ykwwTEtpv5LrLvwAz2DTbJn+UeLUT2jdLF6SYCPiKCbSai0aGNgRksZt5MRwl42QT+CJIezBCfUwWwx+kaFhXgzRMUK1nadKauaXKcCbADUinoaXsjFYw81DSGvTcbtx2+JHPFLhslH/S4E32b4WltdarOvfMsIY+XwvJ/9VmnSlfzy2XbWdAsu1HZSe8RzicDhdcecvEygPKFPezYnZKvfnGnyuYkJ5f+SI4zMbpi3H24eXo4G1vxx56gB6iJyhFr9ABeo8maIpY9DSaRJ+iz/F+PI1JfLKlxlGvuY/+ibj6BbOF0hw=</latexit>

ADMM

Repeat N2 times

Conjugate Gradient

L2 Projection

Dual update

Repeat N3 times


MethodsData:• Train on 4,384 2D slices from 16 3D FSE knee scans (mridata.org)• Validation: 2 separate scans. Test: 2 separate scans (548 slices each)• Average 7 slices to act as noise-free ground-truth• Apply retrospective Poisson disc1 sampling and add noise

Network:• 5 unrolls, 2 ADMM iterations, 6 Conjugate Gradient iterations• Denoising CNN with UNet2 (2ch real/imaginary)• Train with Adam optimizer

Evaluation:• DBP with and without ground-truth, MoDL with ground-truth, PICS3 (l1-wavelet)

[1] M Lustig, MRM 2007. [2] O Ronneberger, MICCAI 2015. [3] M Uecker, BART v0.4.04.

Sampling pattern


NRM

SE

Training epoch

Performance gap

NRMSE vs. training epoch

Noisier updates


Test sliceGround truth Supervised

DBPUnsupervised

DBP PICS

NRMSE: 0.091369 0.067340 0.090781 0.076315

5 unrolls at training5 unrolls at inference

MoDL



DBPUnsupervised

DBP PICS

NRMSE: 0.095818 0.060486 0.072552 0.076315


MoDL



DBPUnsupervised

DBP PICS

NRMSE: 0.113256 0.067340 0.068922 0.076315


MoDL


Test set error


NRMSE on test setNR

MSE

SupervisedDBP

5 unrolls

UnsupervisedDBP

5 unrolls

UnsupervisedDBP

10 unrollsPICS


Ground truth MoDL Supervised DBP Unsupervised DBP PICS

NRMSE: 0.089965 0.060977 0.072900 0.079406

Abs Error


Test sliceGround truth MoDL Supervised

DBPUnsupervised

DBP PICS

NRMSE: 0.089965 0.060977 0.072900 0.079406


Iteration progressCNN DC CNN DC CNN DC CNN DC

MoDL Supervised DBP Unsupervised DBP

A⇤y<latexit sha1_base64="RQj73ozY4nfGdMMb16syD/Gxa/o=">AAAB/3icdVDLSgMxFM34rPU1KrhxEyyCuBimM7Wtu6oblxXsA9paMmmmDc1khiQjlHEW/oobF4q49Tfc+TemL1DRA4HDOfdyT44XMSqVbX8aC4tLyyurmbXs+sbm1ra5s1uXYSwwqeGQhaLpIUkY5aSmqGKkGQmCAo+Rhje8HPuNOyIkDfmNGkWkE6A+pz7FSGmpa+63A6QGnp+cp7cncz5Ku2bOtuxSuVwoQtsqOm7ZLmhy5rjuqQPzlj1BDsxQ7Zof7V6I44BwhRmSspW3I9VJkFAUM5Jm27EkEcJD1CctTTkKiOwkk/wpPNJKD/qh0I8rOFG/byQokHIUeHpynFD+9sbiX14rVn65k1AexYpwPD3kxwyqEI7LgD0qCFZspAnCguqsEA+QQFjpyrK6hPlP4f+k7lh513KuC7nKxayODDgAh+AY5EEJVMAVqIIawOAePIJn8GI8GE/Gq/E2HV0wZjt74AeM9y/aF5ap</latexit>

…




CNN 1… CNN DC




DC 1… CNN DC




CNN 2… CNN DC




DC 2… CNN DC




DC 5… CNN DC




CNN 9… CNN DC




DC 10… CNN DC


ConclusionUnsupervised training across under-sampled data:ü Promising alternative when no ground-truth available✕ Performance cost relative to supervised approach

Deep Basis Pursuit formulation:• Takes advantage of known noise statistics• Robust to number of unrolls


Grant Support:NIH grants R01EB009690, R01EB019241, P41RR09784; Sloan Research Fellowship; Bakar Fellowship; GE Healthcare (research support)

www.jtsense.com

unsupervised deep basis pursuit - icsi | icsistellayu/publication/doc/2019basisismrms… ·...

Documents