-
本文提出了一种基于物理驱动的融合注意力机制的新型卷积网络单像素成像方法. 通过将结合通道与空间注意力机制的模块集成到一个随机初始化的卷积网络中, 利用单像素成像的物理模型约束网络, 实现了高质量的图像重建. 具体来说, 我们将其空间与通道两个维度的注意力机制集成为一个模块, 引入到多尺度U-net卷积网络的各层中, 通过这种方式, 不仅可以利用注意力机制在三维数据立方中提供的关键权重信息, 还充分结合了U-net网络在不同空间频率下强大的特征提取能力. 这一创新方法能够有效捕捉图像细节, 抑制背景噪声, 提升图像重建质量. 实验结果表明, 针对低采样率条件下的图像重建, 与传统非预训练网络相比, 融合注意力机制的方案不仅在直观上图像细节重建的更好, 而且在定量的评价指标(如峰值信噪比和结构相似性)上均表现出显著优势, 验证了其在单像素成像中的有效性与应用前景.This paper presents a novel convolutional neural network-based single-pixel imaging method that integrates a physics-driven fusion attention mechanism. By incorporating a module combining both channel and spatial attention mechanisms into a randomly initialized convolutional network, the method utilizes the physical model constraints of single-pixel imaging to achieve high-quality image reconstruction. Specifically, the spatial and channel attention mechanisms are combined into a single module and introduced into various layers of a multi-scale U-net convolutional network. In the spatial attention mechanism, we extract the attention weight features of each spatial region of the pooled feature map using convolution. In the channel attention mechanism, we pool the three-dimensional feature map into a single-channel signal and input it into a two-layer fully connected network to obtain the attention weight information for each channel. This approach not only leverages the critical weighting information provided by the attention mechanism in the three-dimensional data cube but also fully integrates the powerful feature extraction capabilities of the U-net network across different spatial frequencies. This innovative method effectively captures image details, suppresses background noise, and improves image reconstruction quality. During the experimental phase, we employed the optical path of single-pixel imaging to acquire bucket signals for two target images, “snowflake” and “basket”. By inputting any noise image into a randomly initialized neural network with an attention mechanism, and leveraging the mean square error between simulated and actual bucket signals, we physically constrained the network's convergence. Ultimately, we achieved a reconstructed image that adhered to the physical model. Experimental results demonstrate that, under low sampling rate conditions, the scheme that integrates the attention mechanism not only intuitively reconstructs image details better but also demonstrates significant advantages in quantitative evaluation metrics such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), verifying its effectiveness and potential for application in single-pixel imaging.
-
Keywords:
- Single-pixel imaging /
- attention mechanisms /
- convolutional neural networks /
- image reconstruction
-
图 2 融合注意力机制的U-net卷积神经网络结构示意图. (a) U-net结构的卷积网络; (b) CBAM模块结构总览; (c)空间注意力机制模块; (d)通道注意力机制模块
Fig. 2. Schematic diagram of U-net convolutional neural network structure with integrated attention mechanism. (a) Convolutional neural networks of a U-net architecture; (b) Overall structure of CBAM; (c) Spatial attention module; (d) Channel attention module
图 3 两种重建方案在不同采样率下的结果
Fig. 3. Results of the two reconstruction schemes at different sampling rates
图 6 不同迭代次数下PSNR与损失函数的变化对比. (a)两种方案重建图像的PSNR随迭代次数的变化; (b)我们方案的损失函数在不同初始学习率下随迭代次数的变化
Fig. 6. Comparison of PSNR and loss function under different iterations. (a) The PSNR of the reconstructed images of the two schemes varies with iterations; (b) The loss function of our scheme varies with iterations under different initial learning rates.
-
[1] Kilcullen P, Ozaki T, Liang J 2022 Nat. Commun. 13 7879
Google Scholar
[2] Hahamovich E, Monin S, Hazan Y, Rosenthal A 2021 Nat. Commun. 12 4516
Google Scholar
[3] Shapiro J H 2008 Phys. Rev. A 78 061802
Google Scholar
[4] Ferri F, Magatti D, Gatti A, Bache M, Brambilla E, Lugiato L 2005 Physical review letters 94 183602
Google Scholar
[5] Wang F, Wang C, Deng C, Han S, Situ G 2022 Photon. Res. 10 104
Google Scholar
[6] Pan L, Shen Y, Qi J, Shi J, Feng X 2023 Opt. Express 31 13943
Google Scholar
[7] Song K, Bian Y, Wang D, Li R, Wu K, Liu H, Qin C, Hu J, Xiao L 2024 Laser & Photonics Rev. published online 2401397
[8] Zhao X S, Yu C, Wang C, Li T, Liu B, Lu H, Zhang R, Dou X, Zhang J, Pan J W 2024 Appl. Phys. Lett. 125 211103
Google Scholar
[9] Karpowicz N, Zhong H, Xu J, Lin K I, Hwang J S, Zhang X C 2005 Semicond. Sci. Tech. 20 S293
Google Scholar
[10] Simões M, Vaz P, Cortez A F V 2024. arXiv: 2411.03907[physics.ins-det]
[11] Shwartz S 2021 Science Bulletin 66 857
Google Scholar
[12] Olbinado M P, Paganin D M, Cheng Y, Rack A 2021 Optica 8 1538
Google Scholar
[13] Clemente P, Durán V, Tajahuerce E, Andrés P, Climent V, Lancis J 2013 Opt. Lett. 38 2524
Google Scholar
[14] Jiang W, Yin Y, Jiao J, Zhao X, Sun B 2022 Photon. Res. 10 2157
Google Scholar
[15] Gibson G M, Sun B, Edgar M P, Phillips D B, Hempler N, Maker G T, Malcolm G P A, Padgett M J 2017 Opt. Express 25 2998
Google Scholar
[16] Zhou L, Xiao Y, Chen W 2023 Opt. Express 31 23027
Google Scholar
[17] Xu Y, Lu L, Saragadam V, Kelly K F 2024 Nat. Commun. 15 1456
Google Scholar
[18] Li J, Li X, Yardimci N T, Hu J, Li Y, Chen J, Hung Y C, Jarrahi M, Ozcan A 2023 Nat. Commun. 14 6791
Google Scholar
[19] Li S, Liu X, Xiao Y, Ma Y, Yang J, Zhu K, Tian X 2023 Opt. Express 31 4712
Google Scholar
[20] Zheng P, Dai Q, Li Z, Ye Z, Xiong J, Liu H C, Zheng G, Zhang S 2021 Sci. Adv. 7 eabg0363
Google Scholar
[21] Katz O, Bromberg Y, Silberberg Y 2009 Appl. Phys. Lett. 95 131110
Google Scholar
[22] López-García L, Cruz-Santos W, GarcíaArellano A, Filio-Aguilar P, Cisneros-Martínez J A, Ramos-García R 2022 Opt. Express 30 13714
Google Scholar
[23] Zhang Z, Ma X, Zhong J 2015 Nat. Commun. 6 6225
Google Scholar
[24] Donoho D 2006 IEEE T. Inform. Theory 52 1289
Google Scholar
[25] Duarte M F, Davenport M A, Takhar D, Laska J N, Sun T, Kelly K F, Baraniuk R G 2008 IEEE Signal Proc. Mag. 25 83
Google Scholar
[26] Huang L, Luo R, Liu X, Hao X 2022 Light Sci. Appl. 11 61
Google Scholar
[27] Figueiredo M A T, Nowak R D, Wright S J 2007
[28] pioneers A 2024 Nat. Mach. Intell. 6 1271
Google Scholar
[29] 查文舒, 李道伦, 沈路航, 张雯, 刘旭亮 2022 力学学报 54 543
Google Scholar
Wenshu Z, Daolun L, Luhang S, Wen Z, Xuliang L 2022 Chinese Journal of Theoretical and Applied Mechanics 54 543
Google Scholar
[30] Zhang H, Wang J, Zhang Y, Du X, Wu H, Zhang T 2024 Astronomical Techniques and Instruments 1 1
[31] van Leeuwen C, Podareanu D, Codreanu V, Cai M X, Berg A, Zwart S P, Stoffer R, Veerman M, van Heerwaarden C, Otten S, Caron S, Geng C, Ambrosetti F, Bonvin A M J J 2020. arXiv: 2004.03454[cs.CE]
[32] Barbastathis G, Ozcan A, Situ G 2019 Optica 6 921
Google Scholar
[33] Ruget A, Moodley C, Forbes A, Leach J 2024 Opt. Express 32 41057
Google Scholar
[34] Wetzstein G, Ozcan A, Gigan S, Fan S, Englund D, Soljačić M, Denz C, Miller D A B, Psaltis D 2020 Nat. 588 39
Google Scholar
[35] Lyu M, Wang W, Wang H, Wang H, Li G, Chen N, Situ G 2017 Sci. Rep. 7 17865
Google Scholar
[36] Zhang X, Deng C, Wang C, Wang F, Situ G 2023 ACS Photonics 10 2363
Google Scholar
[37] Li J, Li Y, Li J, Zhang Q, Li J 2020 Opt. Express 28 22992
Google Scholar
[38] Wang F, Wang C, Chen M, Gong W, Zhang Y, Han S, Situ G 2022 Light Sci. Appl. 11 1
Google Scholar
[39] Peng L, Xie S, Qin T, Cao L, Bian L 2023 Opt. Lett. 48 2527
Google Scholar
[40] Liu H, Bian L, Zhang J 2023 Opt. Laser Technol. 157 108600
Google Scholar
[41] Liu X, Han T, Zhou C, Huang J, Ju M, Xu B, Song L 2023 Opt. Express 31 9945
Google Scholar
[42] Hammernik K, Küstner T, Yaman B, Huang Z, Rueckert D, Knoll F, Akçakaya M 2023 IEEE Signal Processing Magazine 40 98
[43] Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G W 2017. arXiv: 1702.08502[cs.CV]
[44] Ulyanov D, Vedaldi A, Lempitsky V 2020 IJCV 128 1867
Google Scholar
[45] Ren W, Nie X, Peng T, Scully M O 2022 Opt. Express 30 47921
Google Scholar
[46] Zhang H, Sindagi V, Patel V M 2020 IEEE Transactions on Circuits and Systems for Video Technology 30 3943
Google Scholar
[47] Lv W, Xiong J, Shi J, Huang Y, Qin S 2021 J. Intell. Manuf. 32 441
Google Scholar
[48] Zhang H, Wang Z, Liu D 2014 IEEE Transactions on Neural Networks and Learning Systems 25 1229
Google Scholar
[49] Baozhou Z, Hofstee P, Lee J, Al-Ars Z 2021. arXiv: 2108.08205[cs.CV]
[50] Karim N, Rahnavard N 2021. arXiv: 2107.01330[cs.CV]
[51] Hoshi I, Shimobaba T, Kakue T, Ito T 2020 Opt. Express 28 34069
Google Scholar
[52] Stollenga M, Masci J, Gomez F, Schmidhuber J 2014. arXiv: 1407.3068[cs.CV]
[53] Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y 2018. arXiv: 1807.02758[cs.CV]
[54] Liao X, He L, Mao J, Xu M 2024 Remote Sensing 16 1688
Google Scholar
[55] Yu W K, Wang S F, Shang K Q 2024 Sensors 24 1012
Google Scholar
[56] Ronneberger O, Fischer P, Brox T 2015. arXiv: 1505.04597[cs.CV]
[57] Lyu M, Wang W, Wang H, Wang H, Li G, Chen N, Situ G 2017 Scientific Reports 7 17865
Google Scholar
[58] Meng Z, Yu Z, Xu K, Yuan X 2021. arXiv: 2108.12654[eess.IV]
[59] Ferri F, Magatti D, Lugiato L A, Gatti A 2010 Phys. Rev. Lett. 104 253603
Google Scholar
[60] Lin J, Yan Q, Lu S, Zheng Y, Sun S, Wei Z 2022 Photonics 9 343
Google Scholar
下载:
计量
- 文章访问数: 441
- PDF下载量: 10
- 被引次数: 0
