PLU-Net: Extraction of multi-scale feature fusion

The Conv block in the original U-Net network consists of two 3x3 convolution operations, two batch normalization operations, and two nonlinear activations (ReLU). However, we notice that this structure has a loss of local information, so we propose the local guided block (LG block, shown as Fig.1), which is divided into two branches and is made up of two 3x3 dilated convolution operations with dilated rates equal to 1 and 3, The results of two dilated convolution operations are then concatenated to enhance feature propagation. Then, to achieve cross-channel information fusion, a 1x1 convolution operation is employed, and nonlinear features are added on the assumption that the size of the feature map remains unchanged. To achieve channel information adaptation, we inserted SE block after LG block to make LS block (shown as Fig.1), similar to PS module. LG block, as comparison to the original convolution block, minimizes the amount of calculation while obtaining richer feature information with a large receptive field thanks to the addition of double branch structure and dilated convolution. It further realizes the adaptation of channel feature information by adding SE block.

ASPP first was proposed in DeepLabv1 Chen et al. (2014) and then improved in DeepLabv2 Chen et al. (2017a) and DeepLabv3 Chen et al. (2017b), as seen in Fig.2. Deeplabv3’s ASPP, which comprises of one 1x1 convolution, three dilated convolutions, and one global average pooling operation, is used as the foundation. The employment of the global average pooling method in up-sampling produces duplicate information and degrades prediction performance, according to experimental verification. As a result, we remove the global average pooling and replace the ordinary void convolution with the depthwise separable convolution Sifre and Mallat (2014), resulting in a reduction of roughly five times the number of parameters compared to the original ASPP. The ASPP’s multi-scale structure allows it to gather more feature information and utilise larger receptive fields, however more feature information may contain duplicated data, lowering the performance of the system.To alleviate the impact of redundant information, we choose to employ the SE block in SENet Hu et al. (2018) to increase the weight of important channel information while decreasing the weight of worthless channel information. The SE block uses squeeze and excitation two processes to learn the importance of each channel’s features, and then strengthens the relevant channel while weakening the idle channel to achieve adaptive feature channel calibration. As a result, a PS module is proposed (shown as Fig.2). After getting additional feature information and employing the large receptive field, this module may combine the advantages of the ASPP module and the SE block to suppress redundant information and strengthen the important channel feature information.

Our PLU-Net, as shown in Fig.3, improves on the original U-Net network architecture by substituting the LS block for the convolution block in the down-sampling and up-sampling pathways. The LS block’s double-branch structure effectively ensures that information loss is minimized at each layer of operation, while feature reuse and successful propagation are ensured by the bigger receptive field. In addition, the U-Net network’s up-sampling and down-sampling periods were reduced from four to three, and a PS module was added at the end of the down-sampling. We may considerably minimize the calculation amount and make the model more lightweight by reducing the number of down-sampling and up-sampling channels in the last layer of the U-Net network, which is 1024. We employ four branches and a greater dilation rate at the same time to obtain a bigger receptive field and consequently richer feature information. The up-sampling step that follows can be completed efficiently. Our network structure can now achieve better performance with fewer parameters and FlOPs because to the combination of these enhancements.

Because both the PS module and the LS block are modular, they can simply be utilized to substitute convolution processes in various network architectures. We designed three models, PU-Net (conv+PS module), LUNet(LS block), and PLU-Net (LS block+PS module), to demonstrate the robustness of our model, in addition to the original U-Net (conv+null here A+B means A for down-sampling and up-sampling pathway and B for PS module, the same below.) We evaluated the models on three biomedical image segmentation datasets in the study.

We use three datasets to compare the U-Net, PU-Net, LU-Net, U-Net++, MultiResUnetIbtehaz and Rahman (2020), DoubleUNetJha et al. (2020), and PLU-Net architectures. We chose U-Net because of its widespread use and relevance in medical image segmentation, as well as the fact that it serves as the foundation for numerous network architectures. The kernel size is set to $3\times 3$ with dilation values of 1 and 3 correspondingly in the LS block, followed by batch normalization and ReLU. Furthermore, the PS module employs depth-wise separable convolution, the results of which are fed into four atrous convolutions with kernel sizes of $3\times 3$ and dilation values of 1, 6, 12, and 18 respectively. The output size is determined by concatenating the results of four atrous convolutions using 1x1 convolution. For the DSB2018 dataset, we used a batch size of 16, four for the ISIC2018 dataset, and two for the CVC dataset. The optimizer is Adam Kingma and Ba (2014), and the two momentum terms are $0.5$ and $0.999$, with a learning rate of 0.0003. The epoch is set to 100, and the loss function is Binary CrossEntropy Loss(BCELoss). All of the experiments are run on four NVIDIA TITAN V GPUs with 12GB of RAM each, using PyTroch Paszke et al. (2019).

To better show the experimental results, we considered several performance metrics, including Precision (PC, Eq.1), Sensitivity (SE, Eq.2), F1-score(F1, which is also known as Dice coefficient, DC, Eq.3) and Jaccard similarity (JS, Eq.4). Variables involved in these formulas are True Positive (TP), False Positive (FP), True Negative (TN), False Negative (FN), Ground Truth (GT), and Segmentation Result (SR).

Table.1 illustrates the results of our experiments using our proposed model and various state-of-the-art U-Net models, such as U-Net++, MultiResUnet, and DoubleUNet, while Table.1 demonstrates the segmentation outcomes of three different biomedical image segmentation tasks. Table.1 shows that our proposed models LU-Net, PU-Net, and PLU-Net are all superior to U-Net. F1 in JS is superior to U-Net among them. On CVC, our model outperforms U-Net by more than 6 and 8 points in F1 and JS, respectively, when compared to U-Net. LU-Net and PU-Net results, on the other hand, reveal that they are superior than U-Net in JS and F1, with PLU-Net outperforming all other models, proving the superiority of LG Block and the capability of PS module. Furthermore, the segmentation results of the three segmentation tasks in Fig.2 show the model’s advantages. In nucleus segmentation, our model performs better on the edges, and in Polyp Segmentation, the model presented in this paper greatly outperforms other models in terms of segmentation performance. Unlike other models with smooth boundary processing, our model has more refined boundary processing in skin lesion segmentation.