-
超快超声定位显微成像(uULM), 突破了传统超声衍射极限, 可实现分辨率远小于发射波长的在体深层微血管精准成像. 通过对微血管中数以万计的运动微泡进行中心点定位和轨迹追踪, uULM技术可重建微血管图像. 通常一张uULM图像需要数十秒甚至数百秒的连续长程图像采集, 这在一定程度上限制了其更广泛的临床应用. 针对这一挑战, 本研究在阐明了超声衍射极限、超分辨率定位理论方法的基础上, 给出了基于傅里叶环相关的分辨率测定原理和实现方法, 并结合传统uULM重建技术, 发展了一种基于生成对抗网络的深度学习超分辨超声成像方法, 以缩减uULM对图像采集时长的依赖, 提高成像速度和成像分辨率. 针对大鼠脑的在体数据分析结果表明, 基于生成对抗网络的超声定位显微技术微血管分辨达到10 μm, 在保持较高超声成像空间分辨率和图像饱和度的同时, 数据采集时间缩减一半, 从而显著降低了uULM对图像数据采集时长的依赖. 相关深度学习模型连接轨迹的计算复杂度较小, 且避免了人工调参以及轨迹筛选, 为加速超分辨率uULM微血流成像和提升uULM成像分辨率提供了一种有效的工具. 相关思路与方法对促进超分辨率uULM成像技术发展具有一定的借鉴意义.
Ultrafast ultrasound localization microscopy (uULM) has broken through the fundamental acoustic diffraction limit by accumulating thousands of sub-wavelength microbubble localisation points and improved the spatial resolution by more than one order of magnitude, which is conducive to clinical diagnosis. By localizing individually injected microbubbles and tracking their movement with a subwavelength resolution, the vasculature microscopy can be achieved with micrometer scale. However, the reconstruction of a uULM image often requires tens or even hundreds of seconds of continuous long-range image acquisition, which limits its clinical application. In order to solve this problem, a generative adversarial network (GAN) based deep learning method is proposed to reconstruct the super-resolution ultrasound localization microscopy. In vivo uULM ultrasound datasets are used to train the network to reconstruct dense vascular networks via localized microbubbles. This approach is validated by using another in-vivo dataset obtained in a rat brain. Results show that GAN based ultrafast ultrasound localization microscopy (GAN-uULM) can resolve micro vessels smaller than 10 μm. Besides, GAN-uULM is able to distinguish small vessels that cannot be continuously reconstructed by using a standard uULM reconstruction method. Saturation parameter based on counting the number of explored pixels is used to evaluate the reconstruction quality. The proposed reconstruction approach reduces the data requirement by half and thus significantly accelerates the uULM imaging. It is illustrasted that for a dataset of 292 s ultrafast acquisition, the saturation of standard uULM image is 33%, while that of GAN-uULM can reach 46%. Fourier ring correlation (FRC) method is utilized to measure the spatial resolution in uULM. Resolutions of the images obtained by standard uULM and GAN-ULM are 7.8 μm and 8.9 μm, respectively. In conclusion, the developed deep learning model is able to connect trajectories with less computational complexity and avoids manual tuning and trajectory screening, providing an effective solution for accelerating ultrasound localization microscopy. -
Keywords:
- super-resolution /
- ultrasonic localization microscopy /
- convolutional neural network /
- generative adversarial network
[1] 钟传钰, 郑元义 2021 中国医学影像技术 37 1799
Google Scholar
Zhong C Y, Zheng Y Y 2021 Chin. J. Med. Imaging Technol. 37 1799
Google Scholar
[2] 王宇森, 陶鸿根 1991 中华内分泌代谢杂志 7 2
Wang Y S, Tao H G 1991 Chin. J. Endocrinol. Metab. 7 2
[3] Chugh B P, Lerch J P, Yu L X, Pienkowski M, Harrison R V, Henkelman R M, Sled J G 2009 Neuroimage 47 1312
Google Scholar
[4] Huang C H, Chen C C V, Siow T Y, Hsu S H S, Hsu Y H, Jaw F S, Chang C 2013 PLoS One 8 e78186
Google Scholar
[5] Hong G, Lee J C, Robinson J T, Raaz U, Xie L M, Huang, N F, Cooke J P, Dai H J 2012 Nat. Med. 18 1841
Google Scholar
[6] Yao J, Wang L, Yang J M, Maslov K I, Wong T T W, Li L, Huang C H, Zou J, Wang L V 2015 Nat. Methods 12 407
Google Scholar
[7] O"Reilly M A, Hynynen K 2013 Med. Phys. 40 110701
Google Scholar
[8] Jiang C, Li Y, Xu K, Ta D 2021 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 68 72
Google Scholar
[9] 臧佳琦, 许凯亮, 韩清见, 陆起涌, 梅永丰, 他得安 2021 70 114304
Google Scholar
Zang J Q, Xu K L, Han Q J, Lu Q Y, Mei Y F, Ta D A 2021 Acta Phys. Sin. 70 114304
Google Scholar
[10] Sui Y H, Yan S Y, Zang J Q, Liu X, Ta D A, Wang W Q, Xu K L 2021 IEEE International Ultrasonics Symposium (IUS)
[11] Sui Y H, Yan S Y, Yu J J, Song J P, Ta D A, Wang W Q, Xu K L 2022 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 69 2425
Google Scholar
[12] Couture O, Bannouf S, Montaldo G, Aubry J F, Fink M 2009 Ultrasound Med. Biol. 35 1908
Google Scholar
[13] Viessmann O M, Eckersley R J, Christensen-Jeffries K, Tang M X, Dunsby C 2013 Phys. Med. Biol. 58 6447
Google Scholar
[14] Desailly Y, Couture O, Fink M, Tanter M 2013 Appl. Phys. Lett. 103 189
Google Scholar
[15] Errico C, Pierre J, Pezet S, Desailly Y, Lenkei Z, Couture O, Tanter M 2015 Nature 527 499
Google Scholar
[16] Fanglue L, Shelton S E, Espíndola D, Rojas J D, Gianmarco P, Dayton P A 2017 Theranostics 7 196
Google Scholar
[17] Xu K L, Guo X Y, Sui Y H, Hingot V, Couture O, Ta D A, Wang W Q 2021 IEEE International Ultrasonics Symposium (IUS)
[18] 郁钧瑾, 郭星奕, 隋怡晖, 宋剑平, 他得安, 梅永丰, 许凯亮 2022 71 174302
Google Scholar
Yu J J, Guo X Y, Sui Y H, Song J P, Ta D A , Mei Y F, Xu K L 2022 Acta Phys. Sin. 71 174302
Google Scholar
[19] Demené C, Robin J, Dizeux A, Heiles B, Pernot M, Tanter M, Perren, Transcranial F 2021 Nat. Biomed. Eng. 5 219
Google Scholar
[20] Huang C, Zhang W, Gong P, Lok U W, Chen S 2021 Phys. Med. Biol. 66 8
Google Scholar
[21] Couture O, Hingot V, Heiles B, Muleki-Seya P, Tanter M 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 1304
Google Scholar
[22] Christensen-Jeffries K, Couture O, Dayton P A, Eldar Y, Hynynen K, Kiessling F, O'Reilly M, Pinton G, Schmitz G, Tang M, Tanter M, van Sloun R J G 2020 Ultrasound Med. Biol. 46 4
Google Scholar
[23] Youn J, Ommen M L, Stuart M B, Thomsen E V, Jensenet J A 2019 IEEE International Ultrasonics Symposium (IUS)
[24] Sloun R J G v , Solomon O, Bruce M, Khaing Z Z, Wijkstra H, Eldar Y C, Mischi M 2021 IEEE Trans. Med. Imaging 40 829
Google Scholar
[25] Liu X, Zhou T, Lu M, Yang Y, He Q, Luo J 2020 IEEE Trans. Med. Imaging 39 3064
Google Scholar
[26] Bar-Zion A, Solomon O, Tremblay-Darveau C, Adam D, Eldar Y. C 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 2365
Google Scholar
[27] Bar-Zion A, Tremblay-Darveau C, Solomon O, Adam D, Eldar Y. C 2016 IEEE Trans. Med. Imaging 36 169
Google Scholar
[28] Nieuwenhuizen R P, Lidke K A, Bates M, Puig D L, Grunwald D, Stallinga S, Rieger B 2013 Nat. Methods 10 557
Google Scholar
[29] Jensen J A, Holm O, Jerisen L J, Bendsen H, Nikolov I S 2005 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 52 881
Google Scholar
[30] Tanter M, Fink M Ultrafast Imaging in Biomedical Ultrasound. 2014 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 61 102
Google Scholar
[31] Montaldo G, Tanter M, Bercoff J, Benech N, Fink M 2009 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 56 489
Google Scholar
[32] Ledoux L, Brands P J, Hoeks A 1997 Ultrason. Imaging 19 1
Google Scholar
[33] Baranger J, Arnal B, Perren F, Baud O, Tanter M, Demené C 2018 IEEE Trans. Med. Imaging 37 1574
Google Scholar
[34] Demené C, Deffieux T, Pernot M, Osmanski B F, Biran V, Gennisson J L, Sieu L A, Bergel A, Franqui S, Correas J M 2015 IEEE Trans. Med. Imaging 34 2271
Google Scholar
[35] Hingot V, Errico C, Tanter M, Couture O 2017 Ultrasonics 77 17
Google Scholar
[36] Heiles B, Chavignon A, Hingot V, Lopez P, Teston E, Couture O 2021 Nat. Biomed. Eng. 6 605
[37] Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y 2014 Adv. Neural Inf. Process. Syst. 27 2672
Google Scholar
[38] Ronneberger O, Fischer P, Brox T 2015 Medical Image Computing and Computer-Assisted Intervention, PT III 9351 234
[39] Nitish S, Geoffrey H, Alex K, Ilya S, Ruslan S 2014 J. Mach. Learn. Res. 15 1929
Google Scholar
[40] Zhao H, Gallo O, Frosio I, Kautz J 2017 IEEE Trans. Comput. Imaging 3 47
Google Scholar
[41] Ouyang W, Aristov A, Hao X, Lelek M, Zimmer C 2018 Nat. Biotechnol. 36 460
Google Scholar
[42] Hingot V, Errico C, Heiles B, Rahal L, Tanter M, Couture O 2019 Sci. Rep. 9 2456
Google Scholar
[43] Hingot V, Chavignon A, Heiles B, Couture O 2021 IEEE Trans. Med. Imaging 40 3812
Google Scholar
-
图 1 超声成像分辨率及微泡的B-mode图像 (a) 两微泡间距恰好为半波长; (b) 两微泡间距在半波长内; (c) 实验测量的微泡点扩散函数及其中心定位(由红点标记)
Fig. 1. The resolution of ultrasound imaging and a B-mode image of microbubbles: (a) The two sources are exactly half a wavelength apart; (b) the two sources are within a half-wavelength distance; (c) microbubbles appearing as point spread function and their localizations.
图 2 uULM常规流程 (a) 超快超声数据在体采集; (b) 杂波滤除; (c) 运动校准; (d) 微泡定位; (e) 微泡追踪; (f) 超分辨率图像重建
Fig. 2. uULM conventional process: (a) In vivo acquisition of ultrafast ultrasound data ; (b) clutter filtering; (c) motion correction; (d) microbubble localization; (e) microbubble tracking; (f) super-resolution image reconstruction.
图 4 小动物用超快超分辨率超声脑成像实验平台
Fig. 4. Ultrafast super-resolution ultrasound brain imaging experimental platform for small animals.
图 5 大鼠脑超分辨率定位显微 (a) 使用标准uULM方法的血管造影; (b) 使用GAN-uULM方法的血管造影
Fig. 5. Ultrasound Localization Microscopy in a rat brain: (a) Angiogram reconstruction using the standard uULM method; (b) angiogram reconstruction using the GAN-uULM method.
图 6 全血管造影的局部特写及其沿白色虚线的强度分布图. 使用标准uULM (a) 和GAN-uULM (b) 分别对体内数据集进行uULM血管造影得到的局部放大图; (c) 绿色和蓝色曲线表示沿水平虚线的强度分布图
Fig. 6. Zoomed-in regions of interest from the whole angiogram and their intensity profiles along the white dashed line. Magnified regions from uULM Angiograms for an in-vivo dataset using the standard uULM method (a) and the GAN-uULM (b); (c) the intensity profiles along a given horizontal dashed line overlaid in green and blue.
图 7 血管饱和度与累积采集时长的关系曲线
Fig. 7. The relationship curves between vascular saturation and cumulative acquisition time.
图 8 不同采集时长对应的超分辨率血流图像 (a), (b) uULM和GAN-uULM在采集时长为40 s时的血流图像; (c), (d) uULM和GAN-uULM在采集时长为80 s时的血流图像
Fig. 8. Super-resolution blood flow images with different cumulative acquisition times: (a), (b) The results of uULM and GAN-uULM with acquisition time of 40 s; (c), (d) the results of uULM and GAN-uULM with acquisition time of 80 s.
图 9 基于FRC曲线的分辨率测量 (a), (b) 将重建结果随机拆分的两个子图像; (c), (d) 2D FFT得到频谱图; (e) FRC曲线, FRC曲线与1/2 bit (黄色)阈值曲线的两个交点被用于测定图像分辨率; (f) 分辨率与累积采集时长的关系曲线
Fig. 9. Resolution measurements based on FRC curves: (a), (b) Two sub-images obtained by randomly splitting the reconstruction results; (c), (d) the frequency spectrograms obtained by 2D FFT; (e) the FRC curves, the two intersections of the FRC curves with the 1/2 bit (yellow) threshold curve are used to determine the image resolution; (f) the relationship curves between resolution and cumulative acquisition time.
-
[1] 钟传钰, 郑元义 2021 中国医学影像技术 37 1799
Google Scholar
Zhong C Y, Zheng Y Y 2021 Chin. J. Med. Imaging Technol. 37 1799
Google Scholar
[2] 王宇森, 陶鸿根 1991 中华内分泌代谢杂志 7 2
Wang Y S, Tao H G 1991 Chin. J. Endocrinol. Metab. 7 2
[3] Chugh B P, Lerch J P, Yu L X, Pienkowski M, Harrison R V, Henkelman R M, Sled J G 2009 Neuroimage 47 1312
Google Scholar
[4] Huang C H, Chen C C V, Siow T Y, Hsu S H S, Hsu Y H, Jaw F S, Chang C 2013 PLoS One 8 e78186
Google Scholar
[5] Hong G, Lee J C, Robinson J T, Raaz U, Xie L M, Huang, N F, Cooke J P, Dai H J 2012 Nat. Med. 18 1841
Google Scholar
[6] Yao J, Wang L, Yang J M, Maslov K I, Wong T T W, Li L, Huang C H, Zou J, Wang L V 2015 Nat. Methods 12 407
Google Scholar
[7] O"Reilly M A, Hynynen K 2013 Med. Phys. 40 110701
Google Scholar
[8] Jiang C, Li Y, Xu K, Ta D 2021 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 68 72
Google Scholar
[9] 臧佳琦, 许凯亮, 韩清见, 陆起涌, 梅永丰, 他得安 2021 70 114304
Google Scholar
Zang J Q, Xu K L, Han Q J, Lu Q Y, Mei Y F, Ta D A 2021 Acta Phys. Sin. 70 114304
Google Scholar
[10] Sui Y H, Yan S Y, Zang J Q, Liu X, Ta D A, Wang W Q, Xu K L 2021 IEEE International Ultrasonics Symposium (IUS)
[11] Sui Y H, Yan S Y, Yu J J, Song J P, Ta D A, Wang W Q, Xu K L 2022 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 69 2425
Google Scholar
[12] Couture O, Bannouf S, Montaldo G, Aubry J F, Fink M 2009 Ultrasound Med. Biol. 35 1908
Google Scholar
[13] Viessmann O M, Eckersley R J, Christensen-Jeffries K, Tang M X, Dunsby C 2013 Phys. Med. Biol. 58 6447
Google Scholar
[14] Desailly Y, Couture O, Fink M, Tanter M 2013 Appl. Phys. Lett. 103 189
Google Scholar
[15] Errico C, Pierre J, Pezet S, Desailly Y, Lenkei Z, Couture O, Tanter M 2015 Nature 527 499
Google Scholar
[16] Fanglue L, Shelton S E, Espíndola D, Rojas J D, Gianmarco P, Dayton P A 2017 Theranostics 7 196
Google Scholar
[17] Xu K L, Guo X Y, Sui Y H, Hingot V, Couture O, Ta D A, Wang W Q 2021 IEEE International Ultrasonics Symposium (IUS)
[18] 郁钧瑾, 郭星奕, 隋怡晖, 宋剑平, 他得安, 梅永丰, 许凯亮 2022 71 174302
Google Scholar
Yu J J, Guo X Y, Sui Y H, Song J P, Ta D A , Mei Y F, Xu K L 2022 Acta Phys. Sin. 71 174302
Google Scholar
[19] Demené C, Robin J, Dizeux A, Heiles B, Pernot M, Tanter M, Perren, Transcranial F 2021 Nat. Biomed. Eng. 5 219
Google Scholar
[20] Huang C, Zhang W, Gong P, Lok U W, Chen S 2021 Phys. Med. Biol. 66 8
Google Scholar
[21] Couture O, Hingot V, Heiles B, Muleki-Seya P, Tanter M 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 1304
Google Scholar
[22] Christensen-Jeffries K, Couture O, Dayton P A, Eldar Y, Hynynen K, Kiessling F, O'Reilly M, Pinton G, Schmitz G, Tang M, Tanter M, van Sloun R J G 2020 Ultrasound Med. Biol. 46 4
Google Scholar
[23] Youn J, Ommen M L, Stuart M B, Thomsen E V, Jensenet J A 2019 IEEE International Ultrasonics Symposium (IUS)
[24] Sloun R J G v , Solomon O, Bruce M, Khaing Z Z, Wijkstra H, Eldar Y C, Mischi M 2021 IEEE Trans. Med. Imaging 40 829
Google Scholar
[25] Liu X, Zhou T, Lu M, Yang Y, He Q, Luo J 2020 IEEE Trans. Med. Imaging 39 3064
Google Scholar
[26] Bar-Zion A, Solomon O, Tremblay-Darveau C, Adam D, Eldar Y. C 2018 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 65 2365
Google Scholar
[27] Bar-Zion A, Tremblay-Darveau C, Solomon O, Adam D, Eldar Y. C 2016 IEEE Trans. Med. Imaging 36 169
Google Scholar
[28] Nieuwenhuizen R P, Lidke K A, Bates M, Puig D L, Grunwald D, Stallinga S, Rieger B 2013 Nat. Methods 10 557
Google Scholar
[29] Jensen J A, Holm O, Jerisen L J, Bendsen H, Nikolov I S 2005 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 52 881
Google Scholar
[30] Tanter M, Fink M Ultrafast Imaging in Biomedical Ultrasound. 2014 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 61 102
Google Scholar
[31] Montaldo G, Tanter M, Bercoff J, Benech N, Fink M 2009 IEEE Trans. Ultrason. Ferroelectr. Freq. Control 56 489
Google Scholar
[32] Ledoux L, Brands P J, Hoeks A 1997 Ultrason. Imaging 19 1
Google Scholar
[33] Baranger J, Arnal B, Perren F, Baud O, Tanter M, Demené C 2018 IEEE Trans. Med. Imaging 37 1574
Google Scholar
[34] Demené C, Deffieux T, Pernot M, Osmanski B F, Biran V, Gennisson J L, Sieu L A, Bergel A, Franqui S, Correas J M 2015 IEEE Trans. Med. Imaging 34 2271
Google Scholar
[35] Hingot V, Errico C, Tanter M, Couture O 2017 Ultrasonics 77 17
Google Scholar
[36] Heiles B, Chavignon A, Hingot V, Lopez P, Teston E, Couture O 2021 Nat. Biomed. Eng. 6 605
[37] Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y 2014 Adv. Neural Inf. Process. Syst. 27 2672
Google Scholar
[38] Ronneberger O, Fischer P, Brox T 2015 Medical Image Computing and Computer-Assisted Intervention, PT III 9351 234
[39] Nitish S, Geoffrey H, Alex K, Ilya S, Ruslan S 2014 J. Mach. Learn. Res. 15 1929
Google Scholar
[40] Zhao H, Gallo O, Frosio I, Kautz J 2017 IEEE Trans. Comput. Imaging 3 47
Google Scholar
[41] Ouyang W, Aristov A, Hao X, Lelek M, Zimmer C 2018 Nat. Biotechnol. 36 460
Google Scholar
[42] Hingot V, Errico C, Heiles B, Rahal L, Tanter M, Couture O 2019 Sci. Rep. 9 2456
Google Scholar
[43] Hingot V, Chavignon A, Heiles B, Couture O 2021 IEEE Trans. Med. Imaging 40 3812
Google Scholar
下载:
计量
- 文章访问数: 6092
- PDF下载量: 193
- 被引次数: 0
