Research on semantic segmentation of remote sensing images based on attention mechanism
-
摘要: 针对遥感图像语义分割技术存在虚警频繁和前景−背景不平衡问题,搭建一种基于注意力机制遥感图像语义分割网络。该网络采用经典编码−解码网络结构,在编码阶段引入双分支自注意力模块,以提升金字塔视觉变换器(pyramid vision transformer, PVT)的性能。双分支自注意力模块不仅保持了PVT捕捉全局信息的能力,还增强了局部细节的捕捉,有效减少了虚警。为解决前景−背景不平衡问题,引入前景−背景平衡模块,通过强化低分辨率下高语义信息对高分辨率下低语义信息的影响,减少背景噪声,提高了分割准确性。该方法在Potsdam数据集的平均交并比为79.1%,优于主流算法。Abstract: Aiming at the problems of frequent false alarms and the foreground-background imbalance in remote sensing image semantic segmentation, an attention mechanism based remote sensing image semantic segmentation network was constructed. The classical encoding-decoding network structure was adopted in the network and a dual-branch self-attention module was introduced in the encoding stage to improve the performance of the pyramid vision transformer (PVT). The dual-branch self-attention module not only maintained the PVT's ability to capture global information, but also enhanced the capture of local details, thereby effectively reducing false alarms. In addition, to solve the problem of foreground-background imbalance, a foreground-background balance module was introduced, which reduced background noise and improved segmentation accuracy by strengthening the influence of high semantic information at low resolution on low semantic information at high resolution. The proposed method achieves an average intersection over union of 79.1% on the Potsdam dataset, outperforming mainstream algorithms.
-
表 1 Potsdam数据集上的实验结果
Table 1. Experimental results on Potsdam dataset
单位:% 网络模型 道路 车辆 植被 建筑 树木 mIoU GLOTS[8] 66.0 84.3 75.2 82.1 76.9 41.1 STransFuse[7] 79.7 85.5 72.9 83.9 73.6 71.4 EMRT[2] 73.9 80.3 83.1 84.8 75.2 73.6 UNetFormer[12] 79.0 86.3 83.5 85.6 76.7 75.2 ST-UNet[9] 76.3 86.3 67.8 86.6 66.3 75.9 CSTUet[4] 78.5 76.7 70.5 87.9 66.6 77.2 AMNet 83.4 86.8 84.2 88.6 78.4 79.1 表 2 消融实验的结果
Table 2. Results of ablation experiment
单位:% 消融策略 原模型的mIoU 变体模型的mIoU AMNet移除DBSM 79.1 72.3 AMNet移除FBBM 79.1 74.5 FPN + FBBM 41.6 48.7 UNet + FBBM 44.3 50.8 -
[1] 梁燕, 易春霞, 王光宇. 基于编解码网络UNet3 + 的遥感影像建筑变化检测[J] . 计算机学报, 2023, 46(8): 1720 − 1733. doi: 10.11897/SP.J.1016.2023.01720 [2] Lu X, Jiang Z G, Zhang H P. Weakly supervised remote sensing image semantic segmentation with pseudo-label noise suppression[J] . IEEE transactions on Geoscience and Remote Sensing, 2024, 62: 5406912. [3] Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J] . IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481 − 2495. doi: 10.1109/TPAMI.2016.2644615 [4] Fan L L, Zhou Y, Liu H M, et al. Combining Swin transformer with UNet for remote sensing image semantic segmentation[J] . IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5530111. [5] Wang W H, Xie E Z, Li X, et al. Pyramid vision transformer: a versatile backbone for dense prediction without convolutions[PP/OL] . (2021-02-24)[2024-10-14] . https://doi.org/10.48550/arXiv.2102.12122. [6] Dong X Y, Bao J M, Chen D D, et al. CSWin transformer: a general vision transformer backbone with cross-shaped windows[PP/OL] . V3. (2022-01-09)[2024-10-25] . https://doi.org/10.48550/arXiv.2107.00652. [7] Gao L, Liu H, Yang M H, et al. STransFuse: fusing Swin transformer and convolutional neural network for remote sensing image semantic segmentation[J] . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 10990 − 11003. doi: 10.1109/JSTARS.2021.3119654 [8] Liu Y H, Zhang Y F, Wang Y, et al. Rethinking transformers for semantic segmentation of remote sensing images[J] . IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5617515. doi: 10.1109/tgrs.2023.3302024 [9] He X, Zhou Y, Zhao J Q, et al. Swin transformer embedding UNet for remote sensing image semantic segmentation[J] . IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 4408715. doi: 10.20944/preprints202511.1233.v1 [10] Zhou P, Xie X Y, Lin Z C, et al. Towards understanding convergence and generalization of AdamW[J] . IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(9): 6486 − 6493. doi: 10.1109/TPAMI.2024.3382294 [11] Li X T, He H, Li X, et al. PointFlow: flowing semantics through points for aerial image segmentation[PP/OL] . (2021-03-11)[2024-10-15] . https://doi.org/10.48550/arXiv.2103.06564. [12] Wang L B, Li R, Zhang C, et al. UNetFormer: a UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery[J] . ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 190: 196 − 214. doi: 10.1016/j.isprsjprs.2022.06.008 -
下载: