基于3D-NAND的神经形态计算

陈阳洋; 何毓辉; 缪向水; 杨道虹

doi:10.7498/aps.71.20220974

摘要

神经形态芯片是一种新兴的AI芯片. 神经形态芯片基于非冯·诺依曼架构, 模拟人脑的结构和工作方式, 相比冯·诺依曼架构的AI芯片, 神经形态芯片在效率和能耗上有显著的优势. 3D-NAND闪存工艺成熟并且存储密度极高, 基于3D-NAND的神经形态芯片受到许多研究者的关注. 然而由于该技术的专利性质, 少有基于3D-NAND神经形态计算的硬件实现. 本文综述了用3D-NAND实现神经形态计算的工作, 介绍了其中前向传播和反向传播的机制, 并提出了目前3D NAND在器件、结构和架构上需要的改进以适用于未来的神经形态计算.

关键词:

Abstract

A neuromorphic chip is an emerging AI chip. The neuromorphic chip is based on non-Von Neumann architecture, and it simulates the structure and working principle of the human brain. Compared with non-Von Neumann architecture AI chips, the neuromorphic chips have significant improvement of efficiency and energy consumption advantages. The 3D-NAND flash memory has the merits of a mature process and ultra-high storage density, and recently it attracted many researchers’ attention. However, owing to the proprietary nature of the technology, there are few hardware implementations. This paper reviews the present research status of neuromorphic computing by using the 3D-NAND flash memory, introduces the forward propagation and backward propagation schemes, and proposes several improvements on the device, structure, and architecture of 3D NAND for neuromorphic computing.

Keywords:

作者及机构信息

1.
华中科技大学, 博士后流动站, 武汉　430074

2.
武汉新芯集成电路制造有限公司, 博士后工作站, 武汉　430205

3.
江城实验室, 武汉　430205

4.
华中科技大学集成电路学院, 武汉　430074

通信作者: 杨道虹, alan_yang@xmcwh.com

Authors and contacts

1.
Post-doctoral Mobile Station, Huazhong University of Science and Technology, Wuhan 430074, China

2.
Post-doctoral Work Station, Wuhan Xinxin Semiconductor Manufacturing Co., Ltd., Wuhan 430205, China

3.
Hubei Yangtze Memory Laboratories, Wuhan 430205, China

4.
School of Integrated Circuit, Huazhong University of Science and Technology, Wuhan 430074, China

Corresponding author: Yang Dao-Hong, alan_yang@xmcwh.com

文章全文 : translate this paragraph

参考文献

[1]	Amodei D, Hernandez D, Sastry G, Clark J, Brockman G, Sutskever I https://openai.com/blog/ai-and-compute/ [2022-4-11]
[2]	Patterson D A, Hennessy J L 2021 Computer Organization and Design RISC-V Edition: the Hardware Software Interface (6th Ed.) (Amsterdam: Morgan Kaufmann) p44
[3]	Gai S https://pensando.io/dennard-scaling-and-other-power-considerations/ [2022-4-11]
[4]	Dally B https://www.cs.colostate.edu/~cs575dl/Sp2015/Lectures/Dally2015.pdf [2022-4-11]
[5]	Drachman D A 2005 Neurology 64 2004 Google Scholar
[6]	Zhang W, Gao B, Tang J, Yao P, Yu S, Chang M F, Yoo H J, Qian H, Wu H 2020 Nat. Electron. 3 371 Google Scholar
[7]	Roy K, Jaiswal A, Panda P 2019 Nature 575 607 Google Scholar
[8]	Khaddam-Aljameh R, Stanisavljevic M, Mas J F, et al. 2021 2021 Symposium on VLSI Technology Kyoto, Japan, June 13–19, 2021 p1
[9]	Narayanan P, Ambrogio S, Okazaki A, et al. 2021 2021 Symposium on VLSI Technology Kyoto, Japan, June 13–19, 2021 p1
[10]	Yang J, Xue X, Xu X, Lv H, Zhang F, Zeng X, Chang M F, Liu M 2020 2020 IEEE Symposium on VLSI Circuits Honolulu, HI, USA, June 16–19, 2020 p1
[11]	Chih Y D, Shih Y C, Lee C F, et al.2020 2020 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, Feb 16–20, 2020 p222
[12]	Liu Y, Su F, Yang Y, Wang Z, Wang Y, Li Z, Li X, Yoshimura R, Naiki T, Tsuwa T, Saito T, Wang Z, Taniuchi K, Yang H 2019 IEEE J. Solid-State Circuits 54 885 Google Scholar
[13]	Dünkel S, Trentzsch M, Richter R, et al. 2017 2017 IEEE International Electron Devices Meeting San Francisco, California, USA, Dec 2–6, 2017, p19.17. 11
[14]	IRDS^TM 2021 International Roadmap for Devices and Systems (2021 Ed.) (IEEE) from https://irds.ieee.org/images/files/pdf/2021/2021IRDS_MM_Tables.xlsx [2022-4-11]
[15]	Liu T Y, Yan T H, Scheuerlein R, et al. 2013 2013 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, Feb. 17–21, 2013 p210
[16]	Chen Y, Li H, Wang X, Zhu W, Xu W, Zhang T 2012 IEEE J. Solid-State Circuits 47 560 Google Scholar
[17]	Rho K, Tsuchida K, Kim D, et al. 2017 2017 IEEE International Solid-State Circuits Conference San Franciso, CA, USA, Feb. 5–9, 2017 p396
[18]	Zwerg M, Baumann A, Kuhn R, et al. 2011 2011 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, Feb. 20–24, 2011 p334
[19]	Takashima D, Nagadomi Y, Ozaki T 2011 IEEE J. Solid-State Circuits 46 681 Google Scholar
[20]	Trentzsch M, Flachowsky S, Richter R, et al. 2016 2016 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 3–7, 2016 p11.15.11
[21]	Lee J W, Na D, Kavala A, et al. 2020 2020 IEEE Symposium on VLSI Circuits Honolulu, HI, United States, June 16–19, 2020 p1
[22]	Mulaosmanovic H, Breyer E T, Dünkel S, Beyer S, Mikolajick T, Slesazeck S 2021 Nanotechnology 32 502002 Google Scholar
[23]	Noguchi H, Ikegami K, Kushida K, et al. 2015 2015 IEEE International Solid-State Circuits Conference San Franciso, CA, USA, February 22–26, 2015 p1
[24]	Sato H, Honjo H, Watanabe T, et al. 2018 2018 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 1–5, 2018 p27.22.21
[25]	Khan A I, Keshavarzi A, Datta S 2020 Nat. Electron. 3 588 Google Scholar
[26]	Dong Q, Kim Y, Lee I, et al. 2017 2017 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, February 5–9 2017 p198
[27]	Cheong W, Yoon C, Woo S, et al. 2018 2018 IEEE International Solid-State Circuits Conference San Francisco, California, USA, February 11–15, 2018 p338
[28]	Liang J, Jeyasingh R G D, Chen H, Wong H P 2011 2011 Symposium on VLSI Technology Kyoto, Japan, June 14–16, 2011 p100
[29]	Wu T F, Le B Q, Radway R, et al. 2019 2019 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, February 17–21, 2019 p226
[30]	Dong Q, Wang Z, Lim J, Zhang Y, Sinangil M E, Shih Y C, Chih Y D, Chang J, Blaauw D, Sylvester D 2019 IEEE J. Solid-State Circuits 54 231 Google Scholar
[31]	Yoon S, Youn Y, Kim S 2015 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems New York, NY, USA, August 24–26, 2015 p1045
[32]	Sandre G D, Bettini L, Pirola A, et al. 2010 2010 IEEE International Solid-State Circuits Conference San Francisco, California, USA, February 7–11, 2010 p268
[33]	Lo C P, Lin W Z, Lin W Y, Lin H T, Yang T H, Chiang Y N, King Y C, Lin C J, Chih Y D, Chang T Y J, Chang M F 2019 IEEE J. Solid-State Circuits 54 584 Google Scholar
[34]	Xue X Y, Jian W X, Yang J G, Xiao F J, Chen G, Xu X L, Xie Y F, Lin Y Y, Huang R, Zhou Q T, Wu J G 2012 2012 Symposium on VLSI Circuits Honolulu, HI, USA, June 13–15, 2012 p42
[35]	Chang M, Shen S, Liu C, Wu C, Lin Y, King Y, Lin C, Liao H, Chih Y, Yamauchi H 2013 IEEE J. Solid-State Circuits 48 864
[36]	Chen J, Chiang R C, Huang H H, Venkataramani G 2012 SIGOPS Oper. Syst. Rev. 45 48 Google Scholar
[37]	Qazi M, Clinton M, Bartling S, Chandrakasan A P 2012 IEEE J. Solid-State Circuits 47 141 Google Scholar
[38]	Breyer E T, Mulaosmanovic H, Trommer J, Melde T, Dünkel S, Trentzsch M, Beyer S, Slesazeck S, Mikolajick T 2020 IEEE J. Electron Devices Soc. 8 748 Google Scholar
[39]	Wang Z, Wu H, Burr G W, Hwang C S, Wang K L, Xia Q, Yang J J 2020 Nat. Rev. Mater. 5 173 Google Scholar
[40]	Lee G H, Hwang S, Yu J, Kim H 2021 Appl. Sci. 11 6703 Google Scholar
[41]	Jang J, Kim H S, Cho W, et al.2009 2009 Symposium on VLSI Technology Kyoto, Japan, June 15–17, 2009 p192
[42]	Wonjoo K, Sangmoo C, Junghun S, Taehee L, Park C, Hyoungsoo K, Juhwan J, Inkyong Y, Park Y 2009 2009 Symposium on VLSI Technology Kyoto, Japan, June 15–17, 2009 p188
[43]	Micheloni R 2016 3D Flash Memories (Dordrecht: Springer Netherlands) p89
[44]	Seo Y T, Kwon D, Noh Y, Lee S, Park M K, Woo S Y, Park B G, Lee J H 2021 IEEE Trans. Electron Devices 68 3801 Google Scholar
[45]	Lee S T, Lim S, Choi N, Bae J H, Kim C H, Lee S, Lee D H, Lee T, Chung S, Park B G, Lee J H 2008 2018 IEEE Symposium on VLSI Technology Honolulu, HI, USA, June 18–22, 2018 p169
[46]	Lee S T, Lim S, Choi N Y, Bae J H, Kwon D, Park B G, Lee J H 2019 IEEE J. Electron Devices Soc. 7 1085 Google Scholar
[47]	Wang P, Xu F, Wang B, Gao B, Wu H, Qian H, Yu S 2019 IEEE Trans. Very Large Scale Integr. VLSI Syst. 27 988 Google Scholar
[48]	Lee S T, Kim H, Bae J H, Yoo H, Choi N Y, Kwon D, Lim S, Park B G, Lee J H 2019 2019 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 7–11, 2019 p38.34.31
[49]	Lue H T, Hsu P K, Wei M L, Yeh T H, Du P Y, Chen W C, Wang K C, Lu C Y 2019 2019 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 7–11, 2019 p38.31.31
[50]	Kim M, Liu M, Everson L, Park G, Jeon Y, Kim S, Lee S, Song S, Kim C H 2019 2019 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 7–11, 2019 p38.33.31
[51]	Kim M, Liu M, Everson L R, Kim C H 2022 IEEE J. Solid-State Circuits 57 625 Google Scholar
[52]	Kang M, Kim H, Shin H, Sim J, Kim K, Kim L S 2022 IEEE Trans. Comput. 71 1291 Google Scholar
[53]	Hsu P K, Du P Y, Lo C R, Lue H T, Chen W C, Hsu T H, Yeh T H, Hsieh C C, Wei M L, Wang K C, Lu C Y 2020 2020 IEEE International Memory Workshop Dresden, Germany, May 17–20, 2020 p1
[54]	Zhou W, Jin L, Jia X, Wang T, Xu P, Zhang A, Huo Z 2022 IEEE Electron Device Lett. 43 374 Google Scholar
[55]	Lee S T, Lee J H 2020 Front. Neurosci. 14 517292 Google Scholar
[56]	Lee S T, Yeom G, Yoo H, Kim H S, Lim S, Bae J H, Park B G, Lee J H 2021 IEEE Trans. Electron Devices 68 3365 Google Scholar
[57]	Kim M K, Kim I J, Lee J S 2021 Sci. Adv. 7 1341 Google Scholar
[58]	Yoon S, Hong S I, Choi G, Kim D, Kim I, Jeon S M, Kim C, Min K 2022 2022 IEEE International Memory Workshop Dresden, Germany, May 15–18, 2022 p 1
[59]	Yoo H N, Back J W, Kim N H, Kwon D, Park B G, Lee J H 2022 2022 IEEE Symposium on VLSI Technology and Circuits Honolulu, HI, USA, June 12–17, 2022 p304

施引文献

图 1 AI模型训练的算力需求和单芯片运算性能^[1]

Fig. 1. Total amount of computing in AI training and single-core chip performance^[1].

下载: 全尺寸图片幻灯片

图 2 限制芯片性能提升的冯·诺依曼瓶颈^[2-4]

Fig. 2. The von Neumann bottleneck limits chip performance promotion^[2-4].

下载: 全尺寸图片幻灯片

图 3 神经形态计算的原理与芯片架构^[6]

Fig. 3. Neuromorphic computing architectures and paradigms^[6].

下载: 全尺寸图片幻灯片

图 4 基于非易失存储器的神经形态计算硬件方案^[7]　(a) 几种作为突触的非易失性存储器件, 其中包括PCM, ReRAM, STT-MRAM, FeRAM和FeFET; (b) 突触和神经元构成crossbar阵列结构用于神经形态计算; (c) 神经形态计算芯片的架构

Fig. 4. Use of non-volatile memory devices as synaptic storage^[7]: (a) Non-volatile memory cell as artificial synapse including PCM, ReRAM, STT-MRAM, FeRAM, and FeFET; (b) the implementation of neuromorphic computation on crossbar array consists of artificial synapses and neurons; (c) typical architecture of the neuromorphic chip.

下载: 全尺寸图片幻灯片

图 5 各种存储器的性能参数对比(数据来源于表 1)

Fig. 5. Performance comparison of various memories (data was extracted from Table 1 and plotted into a radar diagram).

下载: 全尺寸图片幻灯片

图 6 NOR和NAND的电路结构

Fig. 6. Circuit structure diagram of NOR flash and NAND flash.

下载: 全尺寸图片幻灯片

图 7 各种3D-NAND的技术方案^[40]

Fig. 7. A summary of 3D-NAND technologies^[40].

下载: 全尺寸图片幻灯片

图 8 (a) 栅极堆叠的TCAT技术^[41]; (b) 沟道堆叠的VG-NAND技术^[42]

Fig. 8. (a) TCAT technology with gate stack architecture^[41]; (b) VG-NAND technology with channel stack architecture^[42].

下载: 全尺寸图片幻灯片

图 9 (a) CTL单元的结构示意图(以栅极堆叠结构为例)^[43]; (b) 采用ISPP机制调控3D-NAND中CTL器件的阈值电压^[44]; (c) 3D-NAND中CTL器件的模拟电导特性, 即模拟权重调制特性^[45]

Fig. 9. (a) Illustration of typical gate-stack type 3D-NAND^[43]; (b) ISPP modulation of threshold voltage in CTL device^[44]; (c) analog conductivity characteristics of CTL devices in 3D-NAND^[45].

下载: 全尺寸图片幻灯片

图 10 (a) 基于NAND的神经形态计算法则^[46]; (b), (c) 前向传播和反向传播过程中NAND的操作方法^[45]

Fig. 10. (a) Learning rule of software-based and NAND-based neural network^[46]; 2D-NAND operation method in (b) forward and (c) backward propagation^[45].

下载: 全尺寸图片幻灯片

图 11 一种基于3D-NAND的VMM方案^[47]　(a) 3D-NAND的电路结构和VMM的操作方法; (b) 用于3D-NAND层间选通的外围电路结构

Fig. 11. A case of using 3D-NAND for VMM operation^[47]: (a) Circuit diagram and bias scheme of 3D-NAND array architecture for VMM; (b) peripheral circuitry for layer-to-layer selection.

下载: 全尺寸图片幻灯片

图 12 基于2D NAND的二值神经网络BNN^[48]　(a) 相邻的两个CTL器件组成差分对形式的突触, 与输入的信号进行同或运算(XNOR); (b), (c) BNN前向传播中NAND的操作方法以及NAND电路示意图; (d) 采用二值全连接神经网络和二值卷积神经网络分别用于MNIST和CIFAR-10图像库的识别性能

Fig. 12. A synaptic architecture based on 2D NAND for binary neural network (BNN) ^[48]: (a) NAND string structure for XNOR operation, in which two neighboring CTL device constructs a differential pair as one synapse; (b) operation scheme for forward propagation; (c) schematic diagram of synaptic array architecture; (d) the performance of binarized multi-layer and convolutional neural networks for MNIST and CIFAR-10 database recognition task respectively.

下载: 全尺寸图片幻灯片

图 13 采用SLC 3D-NAND实现4 bit精度的卷积神经网络方案^[49]　(a) 卷积神经网络的工作原理示意图(上), 涉及大量的MAC过程(下); (b) 对于多bit权重的MAC过程, 用多个SLC器件构成一个多bit权重; (c) 4 bit精度输入与4 bit精度权重的MAC原理; (d) 卷积核电路的工作原理; (e) VGG 16神经网络的结构图以及用所设计的3D-NAND加速VGG 16的性能

Fig. 13. A case of SLC 3D-NAND for convolution neural network (CNN) with 4-bit resolution^[49]: (a) Flow schematic of CNN; (b) in a MAC array, plural SSLs to stand for multi-bit weight; (c) the arithmetic principle of MAC with 4-bit input and 4-bit weight (4I4W); (d) convolutional core circuit and working principle diagram; (e) schematic diagram of VGG 16 CNN and the simulated performance of 3D-NAND hardware implementation.

下载: 全尺寸图片幻灯片

图 14 基于MLC 3D-NAND的8-bit精度卷积方案^[50,51]　(a) 基于BiCS结构的3D-NAND电路图^[51]; (b) 权重的映射方式, 正、负权重存储在相邻的两个block中^[51]; (c) 2个8 bit精度的输入信号和2个8 bit精度的权重的乘加运算过程^[51]; (d) 基于eNAND的卷积核电路, 有7个block, 28个输入端口, 满足5 × 5卷积核的功能^[51]; (e) 卷积过程的信号时序图^[51]

Fig. 14. A case of MLC 3D-NAND for CNN with 8-bit resolution^[50,51]: (a) Circuit diagram of BiCS type 3D-NAND, the 3D structure can be flattened into a 2D structure^[51]; (b) weight mapping method, positive and negative weight stored in two neighboring blocks^[51]; (c) MAC operation principle of 2 inputs and two weights with 8-bit resolution^[51]; (d) convolutional core circuit diagram with 7 blocks and 28 input ports can be used for 5 × 5 convolution operation^[51]; (e) timing diagram of 5 × 5 convolution operation with 8-bit data and 8-bit weights^[51].

下载: 全尺寸图片幻灯片

图 15 基于3D NAND的S-Flash芯片用于卷积神经网络加速^[52]　(a) 卷积过程示意图; (b) MAC算子的比特对乘、加运算延迟时间的影响(左)和MAC算子的比特对累加运算延迟时间的影响(右); (c) S-Flash芯片的架构; (d) S-Flash中权重分布的示意图, 用16个SLC构成一个差分结构的突触; (e) 通过Booth编码分配权重; (f), (g) 同时操作的SSL和BL增大1倍, MAC次数缩减了1/4

Fig. 15. 3D NAND-based CNN accelerator named as S-Flash^[52]: (a) Convolutional operation of CNN; (b) normalized latency for multiplication, accumulation (left) and MAC operation in various multiplication units (right); (c) overall S-FLASH architecture; (d) overall weight data layout, in which a differential synapse constructed with 16 SLC; (e) weight allocation by Booth coding; (f), (g) double the concurrently operated BLs and SSLs, 4 times faster the MAC operation speed.

下载: 全尺寸图片幻灯片

图 16 S-Flash电路仿真的结果^[52]　(a) S-Flash/S-Flash^*/S-Flash^**运行VGG-16卷积神经网络时的能效; (b) S-Flash^**电路模块的面积对比和运行VGG-16时各电路模块的能耗对比; (c) S-Flash^**与其他芯片的性能参数对比

Fig. 16. Simulation result of S-Flash^[52]: (a) Energy efficiency evaluation result of each VGG-16 layer accelerated by S-Flash/S-Flash^*/S-Flash^**; (b) area and energy breakdown of S-FLASH^**; (c) comparison with the other platform.

下载: 全尺寸图片幻灯片

图 17 输入信号采用十进制编码的3D NAND芯片用于卷积神经网络加速^[53]　(a) 用3D NAND做VMM的操作方法; (b) 卷积核映射的方案; (c) subarray的结构示意图; (d) 芯片的架构示意图; (e) 用3D NAND芯片运行VGG-8神经网络用于CIFAR-10图片库识别时, 各电路模块的能耗对比; (f) 3D NAND与其他芯片的性能对比

Fig. 17. A 3D NAND CNN accelerator with decimal input coding^[53] (a) VMM operation method by using 3D NAND; (b) the mapping method of a CNN kernel; (c) designed subarray configuration; (d) hierarchy of the 3D NAND-based neuromorphic chip architecture; (e) energy breakdown of 3D NAND-based chip on VGG-8 network for the CIFAR-10 dataset; (f) comparison with other chips

下载: 全尺寸图片幻灯片

图 18 基于3D NAND的WTA神经网络用于非监督学习^[54]　(a) 具有差分对结构的3D NAND; (b) CTL器件的模拟权重调制特性; (c) 用3D NAND训练WTA神经网络的过程; (d) WTA神经网络的训练结果

Fig. 18. A 3D NAND-based WTA neural network for unsupervised learning^[54]: (a) Schematic of the differential pair in 3D NAND flash array; (b) analog weight modulation of measured CTL device; (c) the training procedure of WTA neural network using 3D NAND array; (d) stylized letter clustering results before and after training.

下载: 全尺寸图片幻灯片

图 19 3D-NAND中采用脉宽编码实现前向传播^[55]　(a) 前向传播的操作方式、神经网络示意图和WL上读电压的时序图; (b) 左: 前向传播的工作原理示示意图, 即两个标记为正、负权重的突触对构成一个突触, 输出电流相减后通过电容积分转化为电压V_c; 右: 读电压V_read、SSL上输入脉冲V_SSL和V_c的时序图; (c) 3层全连接CNN网络在权重精度为4 bit(QNN)和1 bit(BNN)条件下对CIFAR-10图片数据库的识别性能

Fig. 19. Forward propagation using 3D-NAND with pulse width modulation (PWM) scheme^[55]: (a) Operation scheme of forward propagation, schematic diagram of neural networks, the timing diagram of pulses applied to WLs; (b) Left: schematic diagram of synaptic string array consisting of synapses with positive weight (G⁺) and synapses with negative weight (G^–); Right: timing diagram of V_read, V_SSL, and V_c; (c) simulated classification accuracy of 4-bit QNN and BNN for CIFAR-10 images.

下载: 全尺寸图片幻灯片

图 20 差分对突触阵列中将正、负权重分开放置可实现反向传播^[56]　(a) 前向传播和反向传播过程中对应的突触阵列, 从矩阵运算角度上看互为转置结构; (b) 通常情况下, 差分对结构中的正、负权重在同一个阵列中, 突触阵列与前向传播过程中并非转置的关系, 无法实现反向传播功能; (c) 将正、负权重分开置于不同的阵列中, 可以实现反向传播过程

Fig. 20. Backward propagation can be implemented using a differential synaptic array where positive and negative weights are separated^[56]: (a) The matrix of synapse weight in forward and backward propagation are transposed; (b) synaptic array architecture consisting of two adjacent cells representing G⁺ and G^–, the weights cannot be transposed; (c) synaptic array architecture where G⁺ and G^– weights are separated in different arrays.

下载: 全尺寸图片幻灯片

图 21 基于3D-NAND的前向传播和反向传播的工作原理^[56]　(a) 前向传播中3D-NAND的操作方法; (b) 反向传播中3D-NAND的操作方法; (c) 具有n层全连接突触的神经网络结构; (d) 前向传播和反向传播中WL上选通电压V_read和导通电压V_pass的时序

Fig. 21. Forward and backward propagation using 3D-NAND with PWM scheme^[56]: (a) Synaptic array architecture based on NAND flash memory for forwarding propagation operation; (b) synaptic array architecture based on NAND flash memory for backward propagation operation; (c) schematic of neural networks consisting of n weight layers; (d) timing diagram of V_read and V_pass in forwarding propagation and backward propagation.

下载: 全尺寸图片幻灯片

图 22 器件特性和神经网络性能^[56]　(a) 器件的I_BL-V_BL特性随写入脉冲数量的变化; (b) 图(a)中器件的归一化电导随写脉冲数量的变化; (c) 导通电压V_pass和写电压V_PGM对阈值电压的影响; (d) 初始状态的器件和经历过擦写循环的器件对写入脉冲的响应; (e) 三层全连接神经网络的识别率; (f) 隐藏层数量对识别率的影响

Fig. 22. Device characteristics and neural network performance^[56]: (a) I_BL-V_BL curves with an increasing number of program pulses; (b) normalized conductance responses measured in (a); (c) I_BL-V_WL curves measured in a fresh, V_pass disturbed, and programmed cell; (d) conductance response of fresh and cycled cell; (e) recognition accuracy of 3-layer neural network; (f) recognition accuracy with the number of hidden layers.

下载: 全尺寸图片幻灯片

表 1 几种非易失性存储器的性能参数

Table 1. Benchmark table of performance of emerging memories and typical memories.

	新型存储器					主流存储器
	PCM	ReRAM	STT-MRAM	FeRAM	FeFET	NOR flash	NAND flash
器件结构	1T-1R	1T-1R	1T-1R	1T-1C/ 1T-1FTJ	1T	1T	1T
器件面积F ²	4—40	6—26	9—75	30—40	10—30	10	4
工艺节点/nm	14^[8,9]	14^[10]	22^[11]	130^[12]	22^[13]	40	15(2D) 80(3D)^[14]
芯片容量	16—64 GB (Intel optane)	1 MB— 32 GB^[10,15]	16 kB— 4 GB^[16,17]	16 kB— 128 MB^[18,19]	64 kB— 32 MB^[13,20]	1 MB—2 GB (Micron/Infineon/Macro- nix product manuals)	1 TB^[21]
写电压/V	< 3	< 1	< 2	< 3	～1.5	10—15	15—25
写时间/ns	75^[9]	1—10^[22]	3—14^[23,24]	1—10^[25]	1—10^[25]	10000^[26]	100000^[27]
写功耗/(pJ·bit^–1)	1^[28]	～10^[29]	4.5^[30]	0.1^[25]	1—10 fJ^[25]	49^[26]	～1000^[31]
读时间	12—130 ns^[8,32]	2.9—21 ns^[33,34]	2—26 ns^[30,35]	1—25 ns	1—10 ns	11 ns	3 μs^[27]
读功耗/(pJ·bit^–1)	2.47^[36]	1.76^[29]	0.7^[30]	9.8—19.2^[37]	0.27 fJ/bit^[38]	2.2^[26]	～100^[31]
开关比	> 10⁴	> 10³	> 2	> 2	> 10⁸	> 10⁸	> 10⁸
擦写次数	> 10^{10 [22]}	> 10¹²	> 10¹²	> 10¹²	10⁵—10⁹	< 10⁶	< 10⁵
保持时间/a	> 10	> 10	> 10	> 10	> 10	> 10	> 10
存储比特	2	6^[39]	1	3—4^[39]	2—3	2	4

下载: 导出CSV

表 2 基于3 D-NAND的神经形态计算的各项工作比对

Table 2. A comparison of reviewed works.

	文献工作
	[45]	[47]	[48]	[49]	[50]	[52]	[53]	[54]	[55]	[56]
技术节点	N.A.	N.A.	N.A.	N.A.	65 nm	N.A.	32 nm	N.A.	26 nm	26 nm
芯片容量	N.A.	N.A.	N.A.	64 GB	N.A.	N.A.	1.13 GB	N.A.	N.A.	N.A.
器件类型	N.A.	SLC	SLC	SLC	SLC	SLC	SLC	Analog	MLC	PLC
输入端口	位线	字线	漏端选通管的字线	位线	漏端选通管的字线	位线	位线	字线	位线	位线
输入编码	脉冲幅值	N.A.	二值编码	数字编码	数字编码	数字编码	数字编码	二值编码	脉宽编码	脉宽编码
输入信号精度	模拟	N.A.	1 bit	4 bit	8 bit	8 bit	8 bit	1 bit	模拟	模拟
突触精度	N.A.	1 bit	1 bit	4 bit	8 bit	8 bit	8 bit	N.A.	4 bit/2 bit	6 bit
反向传播方式	另选转置的突触阵列, 误差信号在位线上输入	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.	N.A	N.A.	误差信号施加在源极选通管位线上
神经网络	3层全连接网络	N.A.	4层全连网络、 6层卷积 + 3层全连接	VGG16	LeNet-5	VGG-16	VGG-8	两层全连接网络	6层卷积 + 3层全连接	2—7层全连接网络
神经网络的突触数量	0.3 MB	N.A.	N.A.	138 MB	N.A.	N.A.	110 MB	27	N.A.	N.A.
数据集	MNIST	N.A.	MNIST, CIFAR-10	CIFAR-10	MNIST	N.A.	CIFAR-10	ZVN	CIFAR-10	MNIST
识别率	94.5%	N.A.	98.12%, 87.11%	90%	98.5%	N.A.	N.A.	N.A.	89.38% (4 bit 权重)、87.1% (2 bit权重)	95.65%
能效/ (TOPS·W^–1)	N.A.	N.A.	N.A.	～40	N.A.	0.3	12.95	N.A	N.A.	N.A.

下载: 导出CSV

map

[1]	Amodei D, Hernandez D, Sastry G, Clark J, Brockman G, Sutskever I https://openai.com/blog/ai-and-compute/ [2022-4-11]
[2]	Patterson D A, Hennessy J L 2021 Computer Organization and Design RISC-V Edition: the Hardware Software Interface (6th Ed.) (Amsterdam: Morgan Kaufmann) p44
[3]	Gai S https://pensando.io/dennard-scaling-and-other-power-considerations/ [2022-4-11]
[4]	Dally B https://www.cs.colostate.edu/~cs575dl/Sp2015/Lectures/Dally2015.pdf [2022-4-11]
[5]	Drachman D A 2005 Neurology 64 2004 Google Scholar
[6]	Zhang W, Gao B, Tang J, Yao P, Yu S, Chang M F, Yoo H J, Qian H, Wu H 2020 Nat. Electron. 3 371 Google Scholar
[7]	Roy K, Jaiswal A, Panda P 2019 Nature 575 607 Google Scholar
[8]	Khaddam-Aljameh R, Stanisavljevic M, Mas J F, et al. 2021 2021 Symposium on VLSI Technology Kyoto, Japan, June 13–19, 2021 p1
[9]	Narayanan P, Ambrogio S, Okazaki A, et al. 2021 2021 Symposium on VLSI Technology Kyoto, Japan, June 13–19, 2021 p1
[10]	Yang J, Xue X, Xu X, Lv H, Zhang F, Zeng X, Chang M F, Liu M 2020 2020 IEEE Symposium on VLSI Circuits Honolulu, HI, USA, June 16–19, 2020 p1
[11]	Chih Y D, Shih Y C, Lee C F, et al.2020 2020 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, Feb 16–20, 2020 p222
[12]	Liu Y, Su F, Yang Y, Wang Z, Wang Y, Li Z, Li X, Yoshimura R, Naiki T, Tsuwa T, Saito T, Wang Z, Taniuchi K, Yang H 2019 IEEE J. Solid-State Circuits 54 885 Google Scholar
[13]	Dünkel S, Trentzsch M, Richter R, et al. 2017 2017 IEEE International Electron Devices Meeting San Francisco, California, USA, Dec 2–6, 2017, p19.17. 11
[14]	IRDS^TM 2021 International Roadmap for Devices and Systems (2021 Ed.) (IEEE) from https://irds.ieee.org/images/files/pdf/2021/2021IRDS_MM_Tables.xlsx [2022-4-11]
[15]	Liu T Y, Yan T H, Scheuerlein R, et al. 2013 2013 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, Feb. 17–21, 2013 p210
[16]	Chen Y, Li H, Wang X, Zhu W, Xu W, Zhang T 2012 IEEE J. Solid-State Circuits 47 560 Google Scholar
[17]	Rho K, Tsuchida K, Kim D, et al. 2017 2017 IEEE International Solid-State Circuits Conference San Franciso, CA, USA, Feb. 5–9, 2017 p396
[18]	Zwerg M, Baumann A, Kuhn R, et al. 2011 2011 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, Feb. 20–24, 2011 p334
[19]	Takashima D, Nagadomi Y, Ozaki T 2011 IEEE J. Solid-State Circuits 46 681 Google Scholar
[20]	Trentzsch M, Flachowsky S, Richter R, et al. 2016 2016 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 3–7, 2016 p11.15.11
[21]	Lee J W, Na D, Kavala A, et al. 2020 2020 IEEE Symposium on VLSI Circuits Honolulu, HI, United States, June 16–19, 2020 p1
[22]	Mulaosmanovic H, Breyer E T, Dünkel S, Beyer S, Mikolajick T, Slesazeck S 2021 Nanotechnology 32 502002 Google Scholar
[23]	Noguchi H, Ikegami K, Kushida K, et al. 2015 2015 IEEE International Solid-State Circuits Conference San Franciso, CA, USA, February 22–26, 2015 p1
[24]	Sato H, Honjo H, Watanabe T, et al. 2018 2018 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 1–5, 2018 p27.22.21
[25]	Khan A I, Keshavarzi A, Datta S 2020 Nat. Electron. 3 588 Google Scholar
[26]	Dong Q, Kim Y, Lee I, et al. 2017 2017 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, February 5–9 2017 p198
[27]	Cheong W, Yoon C, Woo S, et al. 2018 2018 IEEE International Solid-State Circuits Conference San Francisco, California, USA, February 11–15, 2018 p338
[28]	Liang J, Jeyasingh R G D, Chen H, Wong H P 2011 2011 Symposium on VLSI Technology Kyoto, Japan, June 14–16, 2011 p100
[29]	Wu T F, Le B Q, Radway R, et al. 2019 2019 IEEE International Solid-State Circuits Conference San Francisco, CA, USA, February 17–21, 2019 p226
[30]	Dong Q, Wang Z, Lim J, Zhang Y, Sinangil M E, Shih Y C, Chih Y D, Chang J, Blaauw D, Sylvester D 2019 IEEE J. Solid-State Circuits 54 231 Google Scholar
[31]	Yoon S, Youn Y, Kim S 2015 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems New York, NY, USA, August 24–26, 2015 p1045
[32]	Sandre G D, Bettini L, Pirola A, et al. 2010 2010 IEEE International Solid-State Circuits Conference San Francisco, California, USA, February 7–11, 2010 p268
[33]	Lo C P, Lin W Z, Lin W Y, Lin H T, Yang T H, Chiang Y N, King Y C, Lin C J, Chih Y D, Chang T Y J, Chang M F 2019 IEEE J. Solid-State Circuits 54 584 Google Scholar
[34]	Xue X Y, Jian W X, Yang J G, Xiao F J, Chen G, Xu X L, Xie Y F, Lin Y Y, Huang R, Zhou Q T, Wu J G 2012 2012 Symposium on VLSI Circuits Honolulu, HI, USA, June 13–15, 2012 p42
[35]	Chang M, Shen S, Liu C, Wu C, Lin Y, King Y, Lin C, Liao H, Chih Y, Yamauchi H 2013 IEEE J. Solid-State Circuits 48 864
[36]	Chen J, Chiang R C, Huang H H, Venkataramani G 2012 SIGOPS Oper. Syst. Rev. 45 48 Google Scholar
[37]	Qazi M, Clinton M, Bartling S, Chandrakasan A P 2012 IEEE J. Solid-State Circuits 47 141 Google Scholar
[38]	Breyer E T, Mulaosmanovic H, Trommer J, Melde T, Dünkel S, Trentzsch M, Beyer S, Slesazeck S, Mikolajick T 2020 IEEE J. Electron Devices Soc. 8 748 Google Scholar
[39]	Wang Z, Wu H, Burr G W, Hwang C S, Wang K L, Xia Q, Yang J J 2020 Nat. Rev. Mater. 5 173 Google Scholar
[40]	Lee G H, Hwang S, Yu J, Kim H 2021 Appl. Sci. 11 6703 Google Scholar
[41]	Jang J, Kim H S, Cho W, et al.2009 2009 Symposium on VLSI Technology Kyoto, Japan, June 15–17, 2009 p192
[42]	Wonjoo K, Sangmoo C, Junghun S, Taehee L, Park C, Hyoungsoo K, Juhwan J, Inkyong Y, Park Y 2009 2009 Symposium on VLSI Technology Kyoto, Japan, June 15–17, 2009 p188
[43]	Micheloni R 2016 3D Flash Memories (Dordrecht: Springer Netherlands) p89
[44]	Seo Y T, Kwon D, Noh Y, Lee S, Park M K, Woo S Y, Park B G, Lee J H 2021 IEEE Trans. Electron Devices 68 3801 Google Scholar
[45]	Lee S T, Lim S, Choi N, Bae J H, Kim C H, Lee S, Lee D H, Lee T, Chung S, Park B G, Lee J H 2008 2018 IEEE Symposium on VLSI Technology Honolulu, HI, USA, June 18–22, 2018 p169
[46]	Lee S T, Lim S, Choi N Y, Bae J H, Kwon D, Park B G, Lee J H 2019 IEEE J. Electron Devices Soc. 7 1085 Google Scholar
[47]	Wang P, Xu F, Wang B, Gao B, Wu H, Qian H, Yu S 2019 IEEE Trans. Very Large Scale Integr. VLSI Syst. 27 988 Google Scholar
[48]	Lee S T, Kim H, Bae J H, Yoo H, Choi N Y, Kwon D, Lim S, Park B G, Lee J H 2019 2019 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 7–11, 2019 p38.34.31
[49]	Lue H T, Hsu P K, Wei M L, Yeh T H, Du P Y, Chen W C, Wang K C, Lu C Y 2019 2019 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 7–11, 2019 p38.31.31
[50]	Kim M, Liu M, Everson L, Park G, Jeon Y, Kim S, Lee S, Song S, Kim C H 2019 2019 IEEE International Electron Devices Meeting San Francisco, CA, USA, December 7–11, 2019 p38.33.31
[51]	Kim M, Liu M, Everson L R, Kim C H 2022 IEEE J. Solid-State Circuits 57 625 Google Scholar
[52]	Kang M, Kim H, Shin H, Sim J, Kim K, Kim L S 2022 IEEE Trans. Comput. 71 1291 Google Scholar
[53]	Hsu P K, Du P Y, Lo C R, Lue H T, Chen W C, Hsu T H, Yeh T H, Hsieh C C, Wei M L, Wang K C, Lu C Y 2020 2020 IEEE International Memory Workshop Dresden, Germany, May 17–20, 2020 p1
[54]	Zhou W, Jin L, Jia X, Wang T, Xu P, Zhang A, Huo Z 2022 IEEE Electron Device Lett. 43 374 Google Scholar
[55]	Lee S T, Lee J H 2020 Front. Neurosci. 14 517292 Google Scholar
[56]	Lee S T, Yeom G, Yoo H, Kim H S, Lim S, Bae J H, Park B G, Lee J H 2021 IEEE Trans. Electron Devices 68 3365 Google Scholar
[57]	Kim M K, Kim I J, Lee J S 2021 Sci. Adv. 7 1341 Google Scholar
[58]	Yoon S, Hong S I, Choi G, Kim D, Kim I, Jeon S M, Kim C, Min K 2022 2022 IEEE International Memory Workshop Dresden, Germany, May 15–18, 2022 p 1
[59]	Yoo H N, Back J W, Kim N H, Kwon D, Park B G, Lee J H 2022 2022 IEEE Symposium on VLSI Technology and Circuits Honolulu, HI, USA, June 12–17, 2022 p304

[1]	方语萱, 夏志良, 杨涛, 周文犀, 霍宗亮. 3D NAND闪存中氟攻击问题引起的字线漏电的改进. , 2024, 73(6): 068502. doi: 10.7498/aps.73.20231557
[2]	杨卫涛, 胡志良, 何欢, 莫莉华, 赵小红, 宋伍庆, 易天成, 梁天骄, 贺朝会, 李永宏, 王斌, 吴龙胜, 刘欢, 时光. 近存计算架构AI芯片中子单粒子效应. , 2024, 73(13): 138502. doi: 10.7498/aps.73.20240430
[3]	方语萱, 杨益, 夏志良, 霍宗亮. 3D NAND闪存中TiN与氧化表面F吸附作用的第一性原理研究. , 2024, 73(12): 128502. doi: 10.7498/aps.73.20240254
[4]	李岩, 陈鑫力, 王伟胜, 石智文, 竺立强. 蛋壳膜电解质栅控氧化物神经形态晶体管. , 2023, 72(15): 157302. doi: 10.7498/aps.72.20230411
[5]	王世场, 卢振洲, 梁燕, 王光义. N型局部有源忆阻器的神经形态行为. , 2022, 71(5): 050502. doi: 10.7498/aps.71.20212017
[6]	李策, 杨栋梁, 孙林锋. 基于二维层状材料的神经形态器件研究进展. , 2022, 71(21): 218504. doi: 10.7498/aps.71.20221424
[7]	江碧怡, 周菲迟, 柴扬. 神经形态阻变器件在图像处理中的应用. , 2022, 71(14): 148504. doi: 10.7498/aps.71.20220463
[8]	蒋子寒, 柯硕, 祝影, 朱一新, 朱力, 万昌锦, 万青. 柔性神经形态晶体管及其仿生感知应用. , 2022, 71(14): 147301. doi: 10.7498/aps.71.20220308
[9]	朱佳雪, 张续猛, 王睿, 刘琦. 面向神经形态感知和计算的柔性忆阻器基脉冲神经元. , 2022, 71(14): 148503. doi: 10.7498/aps.71.20212323
[10]	张宇琦, 王俊杰, 吕子玉, 韩素婷. 应用于感存算一体化系统的多模调控忆阻器. , 2022, 71(14): 148502. doi: 10.7498/aps.71.20220226
[11]	王童, 温娟, 吕康, 陈健中, 汪亮, 郭新. 仿生生物感官的感存算一体化系统. , 2022, 71(14): 148702. doi: 10.7498/aps.71.20220281
[12]	温新宇, 王亚赛, 何毓辉, 缪向水. 忆阻类脑计算. , 2022, 71(14): 140501. doi: 10.7498/aps.71.20220666
[13]	沈柳枫, 胡令祥, 康逢文, 叶羽敏, 诸葛飞. 光电神经形态器件及其应用. , 2022, 71(14): 148505. doi: 10.7498/aps.71.20220111
[14]	单旋宇, 王中强, 谢君, 郑嘉慧, 徐海阳, 刘益春. 面向感存算一体化的光电忆阻器件研究进展. , 2022, 71(14): 148701. doi: 10.7498/aps.71.20220350
[15]	任宽, 张珂嘉, 秦溪子, 任焕鑫, 朱守辉, 杨峰, 孙柏, 赵勇, 张勇. 基于忆容器件的神经形态计算研究进展. , 2021, 70(7): 078701. doi: 10.7498/aps.70.20201632
[16]	徐威, 王钰琪, 李岳峰, 高斐, 张缪城, 连晓娟, 万相, 肖建, 童祎. 新型忆阻器神经形态电路的设计及其在条件反射行为中的应用. , 2019, 68(23): 238501. doi: 10.7498/aps.68.20191023
[17]	贺黎明, 曹伟, 陈学谦, 朱云霞. 氦原子1snd(n=4—11)组态下1D—3D谱项分裂值的计算. , 2005, 54(11): 5077-5081. doi: 10.7498/aps.54.5077
[18]	牟致栋, 魏琦瑛. MoⅩⅣ—RuⅩⅥ离子的3d104s—3d94s4p跃迁谱线波长和振子强度的计算. , 2004, 53(6): 1742-1748. doi: 10.7498/aps.53.1742
[19]	白丽华, 张庆刚, 刘新国. D+CD4→CD3+D2反应的四维量子散射计算. , 2003, 52(11): 2774-2780. doi: 10.7498/aps.52.2774
[20]	朱元贞, 潘少华. V3X系d带相对移动量的配位场方法的计算. , 1983, 32(3): 411-416. doi: 10.7498/aps.32.411

计量

文章访问数: 12268
PDF下载量: 299
被引次数: 0

姓名
邮箱
手机号码
标题
留言内容
验证码

搜索

留言板