-
云是地基红外高光谱仪器探测大气的重要干扰源, 有效云检测不可或缺. 水汽干扰和高云识别精度低是云检测面临的两个关键挑战. 本文利用大气红外光谱探测仪(ASSIST)在云南丽江、西藏自治区墨脱和西藏自治区日土的观测数据, 分析了晴空和有云条件下的光谱特征差异, 并据此提出了一种光谱特征增强的机器学习云检测方法. 结合同步观测的激光雷达、气象站及全天空成像仪数据, 系统评估了该方法在不同相对湿度(RH)和不同云底高度(CBH)条件下的检测性能. 实验结果表明: 该方法与激光雷达检测结果的一致性高达97.61%. 在不同RH条件下, 该方法精度均优于使用原始光谱特征的方法, 尤其在RH > 70%时, 对晴空光谱的识别精度提升明显, 从86.01%提高至91.89%. 同样, 在不同CBH条件下, 新方法也展现出优于使用原始光谱特征方法的性能, 特别在识别3 km < CBH 5 km的中云和CBH > 5 km的高云时, 精度提升尤为明显. 当3 km < CBH 5 km时, 精度从95.45%提升至98.64%; 当CBH > 5 km时, 精度从87.5%提升至91.67%.Clouds exert a significant influence on infrared radiation, making cloud detection a crucial step in the application of infrared hyperspectral data. Ground-based infrared hyperspectrometers can measure downward atmospheric thermal radiation with high temporal resolution; however, their spectral radiance measurements are strongly affected by atmospheric conditions. In particular, water vapor interference and the limited accuracy in high-cloud identification constitute two key challenges for ground-based infrared hyperspectral cloud detection. Traditional threshold-based cloud detection methods are difficult to adapt to different locations and dynamically changing atmospheric conditions,while machine learning methods can achieve cloud detection with higher accuracy, greater robustness, and improved automation. Building on the advantages of machine learning, observational data from the atmospheric sounder spectrometer by infrared spectral technology (ASSIST), collected at Lijiang (Yunnan), Motuo (Xizang Autonomous Region), and Ritu (Xizang Autonomous Region) in China, are used to analyze the spectral differences between sunny and cloudy conditions in this study. Based on these differences, a spectral feature enhancement-driven machine learning method for cloud detection is proposed. Finally, by incorporating synchronous observations from lidar, meteorological stations, and all-sky imagers, the proposed method is systematically evaluated under different relative humidity (RH) and cloud base height (CBH) conditions. The experimental results show that the consistency between the results obtained by the proposed method and lidar-based detection is as high as 97.61%. Under different RH conditions, the proposed method outperforms the method based on original spectral features. Notably, when ${\text{RH}} > 70{\text{%}} $, the accuracy of clear-sky spectral identification improves significantly: increasing from 86.01% to 91.89%. Similarly, under different CBH conditions, the proposed method also exhibits superior performance compared with the method in which original spectral features are used. In particular, the accuracy improvements are especially notable when identifying mid-level clouds with ${\text{3 km}} < {\text{CBH}} \leqslant 5{\text{ km}}$, as well as high-level clouds with ${\text{CBH}} > 5{\text{ km}}$. When ${\text{3 km}} < {\text{CBH}} \leqslant 5{\text{ km}}$, the accuracy increases from 95.45% to 98.64% and when ${\text{CBH}} > 5{\text{ km}}$, the accuracy improves from 87.5% to 91.67%. The proposed method significantly enhances the automation and accuracy of cloud detection in ground-based infrared hyperspectral radiance data, thereby providing higher-quality fundamental datasets for supporting subsequent applications such as radiative transfer simulation, remote sensing parameter retrieval, and data assimilation in numerical weather prediction (NWP) models.
-
Keywords:
- ground-based infrared hyperspectroscopy /
- remote sensing /
- machine learning /
- cloud detection
-
图 2 ASSIST测得的几种错误光谱(橙色线条)和正确光谱(绿色线条) (a), (b), (c), (d)分别代表采集的4种错误光谱; (e), (f) 分别代表晴空和多云采集的正确光谱
Fig. 2. Several erroneous spectra (orange lines) and correct spectra (green lines) measured by ASSIST: (a), (b), (c), (d) respectively represent the four types of erroneous spectra collected; (e), (f) respectively represent the correct spectra collected under clear sky and cloudy conditions.
图 3 三个地点晴空和多云光谱特征 (a) 540—1800 cm–1波段晴空和多云光谱特征; (b) 920—970 cm–1波段新增的晴空和多云光谱特征的局部放大图
Fig. 3. The spectral characteristics of clear sky and cloudy conditions in three locations: (a) The spectral characteristics of clear sky and cloudy conditions in the 540–1800 cm–1 band; (b) a local magnified view of the newly added spectral characteristics of clear sky and cloudy conditions in the 920–970 cm–1 band.
图 5 表2中用于区分有云和无云晴空场景的不同特征重要性排序结果 (a) 使用原始12个特征重要性排序结果; (b) 新增8个特征后重要性排序结果
Fig. 5. Importance ranking results of different features used to distinguish between cloudy and cloud-free clear sky scenes in Table 2: (a) The importance ranking results using the original 12 features; (b) the importance ranking results after adding 8 new features.
图 6 测试集中三个地点数据在不同RH条件下出现的概率
Fig. 6. The probability of the data from the three locations in the test set appearing under different RH conditions.
图 7 不同RH下, ASSIST在高美古所测晴空和多云光谱的特征 (a) 540—1800 cm–1波段晴空和多云光谱特征; (b) 920—970 cm–1波段新增的晴空和多云光谱特征的局部放大图
Fig. 7. The spectral characteristics of clear sky and cloudy sky measured by ASSIST in Gao Meigu under different RH conditions: (a) The spectral characteristics of clear sky and cloudy conditions in the 540–1800 cm–1 band; (b) a local magnified view of the newly added spectral characteristics of clear sky and cloudy conditions in the 920–970 cm–1 band.
图 8 ASSIST在日土测得的晴空和不同CBH下多云光谱以及激光雷达探测的云和气溶胶总消光系数结果 (a) ASSIST测得的晴空和多云光谱; (b) 激光雷达探测的云和气溶胶总消光系数结果
Fig. 8. The results of the ASSIST measurements of clear sky and cloudy spectra, as well as the total extinction coefficients of clouds and aerosols detected by the lidar under different CBH conditions: (a) The clear sky and cloudy spectra measured by ASSIST; (b) the total extinction coefficient results of clouds and aerosols detected by lidar.
图 9 测试集中3个地点数据在不同CBH条件下出现的概率
Fig. 9. The probability of the data from the three locations in the test set appearing under different cloud base height conditions.
图 10 在2025年3月25号不同时刻, ASSIST在墨脱测得的晴空和薄雾光谱 (a) ASSIST测得的光谱; (b) 激光雷达探测的云和气溶胶总消光系数结果
Fig. 10. The spectra of clear sky and mist measured by ASSIST at different times in Motuo on March 25, 2025: (a) The spectra measured by ASSIST; (b) the total extinction coefficient results of clouds and aerosols detected by lidar.
图 11 在2025年3月25号3个薄雾时刻, 全天空成像仪拍摄的图像 (a), (b), (c)分别代表北京时间 12:00:03, 13:01:11, 14:04:26
Fig. 11. The images captured by the all-sky imager at three mist moments on March 25, 2025: (a), (b), (c) Represent Beijing time 12:00:03, 13:01:11, 14:04:26, respectively.
图 12 在2025年3月24号不同时刻, ASSIST在墨脱测得的晴空和厚雾光谱 (a) ASSIST测得的光谱; (b) 激光雷达探测的云和气溶胶总消光系数结果
Fig. 12. The spectra of clear sky and thick fog measured by ASSIST at different times in Motuo on March 24, 2025: (a) The spectra measured by ASSIST; (b) the total extinction coefficient results of clouds and aerosols detected by lidar.
图 13 在2025年3月24号3个厚雾时刻, 全天空成像仪拍摄的图像 (a), (b), (c)分别代表北京时间 03:12:08, 05:46:59, 08:21:50
Fig. 13. The images captured by the all-sky imager at three thick fog moments on March 24, 2025: (a), (b), (c) Represent Beijing time 03:12:08, 05:46:59, 08:21:50, respectively.
表 1 三个地点的晴空和多云样本数量
Table 1. The number of clear-sky and cloudy samples at three locations.
地点 晴空样本 多云样本 海拔/km 观测时间 丽江高美古
天文台3357 2826 3.23 2024.03.20—
2024.05.04墨脱气象
观测站1584 3641 0.76 2024.11.29—
2024.12.19
2025.03.15—
2025.03.28日土阿里荒漠环
境综合观测站4052 1543 4.23 2025.05.27—
2025.06.15总计 8993 8010 
下载: 导出CSV
表 2 用于区分多云和晴空数据的20个选定特征(前12个特征代表原始特征, 后8个特征代表新增特征)
Table 2. Twenty selected features used to distinguish between cloudy and clear-sky data (the first 12 features represent the original features and the last 8 features represent the added features).
编号 特征 1 740—760 cm–1波段辐亮度的斜率 2 740—760 cm–1波段辐亮度的截距 3 780—920 cm–1波段辐亮度的斜率 4 780—920 cm–1波段辐亮度的截距 5 1000—1040 cm–1波段辐亮度斜率 6 1000—1040 cm–1波段辐亮度截距 7 1050—1070 cm–1波段辐亮度斜率 8 784.6 cm–1通道辐射与781.7—782.6 cm–1波段平均辐射之间的比率 9 791.8 cm–1通道辐射与789.4—790.4 cm–1波段平均辐射之间的比率 10 1175 cm–1和1170 cm–1通道辐射之间的比率 11 1187 cm–1和1184 cm–1通道辐射之间的比率 12 1198 cm–1和1195 cm–1通道辐射之间的比率 13 925.8524 cm–1通道辐亮度 14 948.9987 cm–1通道辐亮度 15 951.892 cm–1通道辐亮度 16 962.5007 cm–1通道辐亮度 17 925.8524 cm–1 和 925.3702 cm–1 通道辐射之间的比率 18 948.9987 cm–1 和948.5165 cm–1通道辐射之间的比率 19 951.892 cm–1和951.4098 cm–1通道辐射之间的比率 20 962.5007 cm–1和962.0185 cm–1通道辐射之间的比率
下载: 导出CSV
表 3 云检测使用的训练集和测试集样本数
Table 3. The number of samples in the training set and test set used for cloud detection.
数据集 晴天样本数 多云样本数 总计 训练集(70%) 6290 5604 11894 测试集(30%) 2703 2406 5109
下载: 导出CSV
表 4 提出的算法和激光雷达云检测结果的混淆矩阵
Table 4. The confusion matrix of the proposed algorithm and lidar cloud detection results.
激光雷达探测 有云 晴空 云检测算法
(ASSIST)有云 TP
(True positive)FP
(False positive)晴空 FN
(False negative)TN
(True negative)
下载: 导出CSV
表 5 使用原始特征排序后不同特征个数对应的云检测结果
Table 5. Cloud detection results with different numbers of features after sorting the original features.
特征个数 PC/% TPR/% TNR/% 1 95.01 90.90 98.67 2 95.49 93.81 97.37 3 92.88 95.84 90.23 4 94.85 96.97 92.97 5 95.56 97.26 94.04 6 85.71 97.17 75.51 7 79.43 97.22 63.60 8 86.36 97.63 76.32 9 76.59 97.67 57.82 10 76.57 97.67 57.79 11 78.74 97.88 61.71 12 81.64 98.09 67.00
下载: 导出CSV
表 6 使用新增特征排序后不同特征个数对应的云检测结果
Table 6. Cloud detection results with different numbers of features after sorting the newly added features.
特征个数 PC/% TPR/% TNR/% 1 95.30 91.98 98.26 2 94.73 94.43 95.01 3 94.72 94.43 94.97 4 96.50 94.72 98.08 5 96.24 94.80 97.52 6 96.20 96.30 96.12 7 96.46 96.76 96.19 8 96.54 97.09 96.04 9 96.56 98.09 95.19 10 97.61 98.21 97.08 11 82.60 97.38 69.44 12 95.13 97.76 92.79 13 96.81 97.42 96.26 14 96.81 97.42 96.26 15 96.59 97.42 95.86 16 88.88 97.96 80.80 17 88.49 97.88 80.13 18 80.86 98.21 65.41 19 91.41 97.76 85.76 20 91.41 97.80 85.72
下载: 导出CSV
表 7 不同RH下测试集中三个地点总的晴空和多云样本数(括号中的百分比表示测试集中所选RH范围内的数据与总测试集数据之间的比例)
Table 7. The total number of clear-sky and cloudy samples at the three locations in the test set under different RH conditions (The percentages in parentheses indicate the proportion of data within the selected RH range in the test set to the total test set data).
不同水汽 测试集
晴空样本测试集
多云样本总计 ${\text{RH}} \leqslant 30{\text{%}} $ 1883 1060 2943(57.6%) $30{\text{%}} < {\text{RH}} \leqslant 50{\text{%}} $ 250 79 329(6.4%) $50{\text{%}} < {\text{RH}} \leqslant 70{\text{%}} $ 327 158 485(9.5%) ${\text{RH}} > 70{\text{%}} $ 243 1109 1352(26.5%)
下载: 导出CSV
表 8 ASSIST和激光雷达在不同RH条件下云检测结果的一致性
Table 8. Consistency of cloud detection results by ASSIST and lidar under different RH conditions.
不同RH 方法 PC/
%TPR/
%TNR/
%FPR/
%FNR/
%${\text{RH}} \leqslant 30{\text{%}} $ 原始方法 94.33 94.53 94.21 5.79 5.47 新方法 97.93 96.89 98.51 1.49 3.11 $30{\text{%}} < {\text{RH}} \leqslant 50{\text{%}} $ 原始方法 94.53 93.67 94.80 5.20 6.33 新方法 96.66 94.94 97.20 2.80 5.06 $50{\text{%}} {\text{ < RH}} \leqslant 70{\text{%}} $ 原始方法 98.76 99.37 98.47 1.53 0.63 新方法 99.58 99.40 100.00 0 0.60 ${\text{RH}} > 70{\text{%}} $ 原始方法 97.34 99.82 86.01 13.99 0.18 新方法 98.82 99.83 91.89 8.11 0.17
下载: 导出CSV
表 9 不同CBH下测试集中3个地点总的多云样本数(括号中的百分比表示测试集中所选CBH范围内的数据与总测试集数据之间的比例)
Table 9. The total number of cloudy samples at the three locations in the test set under different cloud base height conditions (The percentages in parentheses indicate the proportion of data within the selected cloud base height range in the test set to the total test set data).
不同CBH 测试集多云样本 ${\text{CBH}} \leqslant 1{\text{ km}}$ 1196(49.69%) $1{\text{ km < CBH}} \leqslant {\text{3 km}}$ 494(20.52%) $3{\text{ km < CBH}} \leqslant 5{\text{ km}}$ 646(26.86%) ${\text{CBH}} > 5{\text{ km}}$ 70(2.93%)
下载: 导出CSV
表 10 ASSIST和激光雷达在不同CBH条件下云检测结果的一致性
Table 10. Consistency of cloud detection results by ASSIST and lidar under different cloud base height conditions.
不同CBH 方法 PC/% TPR/% FNR/% ${\text{CBH}} \leqslant 1{\text{ km}}$ 原始方法 98.53 98.53 1.47 新方法 99.26 99.26 0.74 $ 1{\text{ km}} < {\text{CB}}H \leqslant 3{\text{ km}} $ 原始方法 96.43 96.43 3.57 新方法 97.62 97.62 2.38 ${\text{3 km}} < {\text{CBH}} \leqslant 5{\text{ km}}$ 原始方法 95.45 95.45 4.55 新方法 98.64 98.64 1.36 ${\text{CBH}} > 5{\text{ km}}$ 原始方法 87.50 87.50 12.5 新方法 91.67 91.67 8.33
下载: 导出CSV
-
[1] Govender M, Chetty K, Bulcock H 2007 Water S. A. 33 145
[2] Vorovencii I 2010 Bull. Transilvania Univ. Bras. II: For. Wood Ind. Agric. Food Eng. 2 51
[3] Shimoda H, Ogawa T 2000 Adv. Space Res. 25 937
Google Scholar
[4] Aumann H H, Miller C R 2002 Proc. SPIE 4483 332
[5] Clerbaux C, Hadji-Lazaro J, Turquety S, George M, Coheur P F, Hurtmans D, Wespes C, Herbin H, Blumstein D, Tourniers B, Phulpin T 2007 Space Res. Today 168 19
Google Scholar
[6] Andrew S, Nigel A, William B, Amy D 2015 Atmos. Sci. Lett. 16 260
Google Scholar
[7] Qi C L, Wu C Q, Hu X C, Xu H L, Lee L, Zhou F, Gu M J, Yang T H, Shao C Y, Yang Z D, Zhang P 2020 IEEE Trans. Geosci. Remote Sens. 58 4335
Google Scholar
[8] Yang J, Zhang Z Q, Wei C Y, Lu F, Guo Q 2017 Bull. Am. Meteorol. Soc. 98 1637
Google Scholar
[9] Knuteson R O, Revercomb H E, Best F A, Ciganovich N C, Dedecker R G, Dirkx T P, Ellington S C, Feltz W F, Garcia R K, Howell H B, Smith W L, Short J F, Tobin D C 2004 J. Atmos. Oceanic Technol. 21 1763
Google Scholar
[10] Rochette L, Smith W, Howard M, Bratcher T 2009 SPIE 7457 002
[11] Turner D D, Löhnert U 2014 J. Appl. Meteorol. Climatol. 53 752
Google Scholar
[12] Mariani Z, Strong K, Palm M, Lindenmaier R, Adams C, Zhao X, Savastiouk V, McElroy C T, Goutail F, Drummond J R 2013 Atmos. Meas. Tech. 6 1549
Google Scholar
[13] Seo J, Choi H, Oh Y 2022 Remote Sens. 14 407
Google Scholar
[14] Ye J, Liu L, Yang W Y, Ren H 2022 IEEE Geosci. Remote Sens. Lett. 19 1
[15] Wang Y, Xiong W, Ye H H, Shi H L, Wang X H, Li C, Wu S C, Cheng C 2025 Remote Sens. 17 1440
Google Scholar
[16] Wang Y, Ye H H, Shi H L, Wang X H, Li C, Sun E C, An Y, Wu S C, Xiong W 2024 J. Quant. Spectrosc. Radiat. Transfer 326 109118
Google Scholar
[17] Mcnally A, Watts P A 2003 Q. J. R. Meteorol. Soc. 129 3411
Google Scholar
[18] Bauer P, Auligné T, Bell W, Geer A, Guidard V, Heilliette S, Kazumori M, Kim M J, Liu E, McNally A, Macpherson B, Okamoto K, Renshaw R, Riishøjgaard L P 2011 Q. J. R. Meteorol. Soc. 137 1934
Google Scholar
[19] Löhnert U, Turner D, Crewell S 2009 J. Appl. Meteorol. Climatol. 48 1017
Google Scholar
[20] Li C, Ma J J, Yang P, Li Z Q 2019 J. Quant. Spectrosc. Radiat. Transfer 222 196
[21] Cho J S, Goo T Y, Shin J 2015 Korean Soc. Remote Sens. 31 137
Google Scholar
[22] Zhang Q, Yu Y, Zhang W M, Luo T L, Wang X 2019 Remote Sens. 11 3035
Google Scholar
[23] Luo T L, Zhang W M, Yu Y, Feng M, Duan B H, Xing D 2019 Int. J. Remote Sens. 40 6530
Google Scholar
[24] Shi H X, Yu Y, Zhang W M, Ma G, Zhang Q, Luo T L, Huang Q B 2021 Proceedings of the 2021 International Conference on Computer Information Science and Artificial Intelligence (CISAI) Kunming, China, September 17–19, 2021 p107
[25] Liu L, Ye J, Li S L, Hu S, Wang Q 2022 Remote Sens. 14 2589
Google Scholar
[26] Michaud-Belleau V, Gaudreau M, Lacoursière J, Boisvert É, Ravelomanantsoa L, Turner D, Rochette L 2025 EGUsphere: 2024-3617 [atmospheric sciences]
[27] 赵强, 苏红超, 易明建, 余东升, 徐赤东 2021 中国激光 48 201001
Zhao Q, Su H C, Yi M J, Yu D S, Xu C D 2021 Chin. J. Lasers 48 2010001
[28] TURNER D 2007 J. Geophys. Res. Atmos. 112 D15
[29] Ishida H, Oishi Y, Morita K, Moriwaki K, Nakajima T 2018 Remote Sens. Environ. 205 390
Google Scholar
[30] Wang X P, Zhang F, Kung H T, Johoson V C, Latif A 2020 Int. J. Remote Sens. 41 953
Google Scholar
下载:
计量
- 文章访问数: 463
- PDF下载量: 9
- 被引次数: 0
