Edge-Aware Mirror Network for Camouflaged Object Detection
^††thanks: *Corresponding author. This work was supported in part by the National Natural Science Foundation of China under the Grant 41927805.

Figure 2 shows the overview of the proposed EAMNet, which consists of three kinds of key components including the bifurcated backbone, the edge detection branch, and the segmentation branch. The edge detection branch consists mainly of the SEA module, which aggregates multi-level edge features under the guidance from segmentation branch. The segmentation branch consists mainly of the EIA module, which aggregates multi-level segmentation features under the the guidance from edge detection branch. Both the two branches adopt the coarse-to-fine strategy under the supervision of edge map and ground truth map, respectively, which means that the prediction in two branches is progressively optimized from low to high resolution.

Specifically, given an input image I, we first adopt the Res2Net-50 [17] as backbone to extract features at four levels, which can be denoted as $F=\{f_{i},i=1,2,3,4\}$. Then, we fed F into channel reduce modules containing a $1\times 1$ convolutional layer to extract multi-level edge features with the channel size of 64 denoted as $F_{e}=\{f^{e}_{i},i=1,2,3,4\}$ and multi-level segmentation features denoted as $F_{s}=\{f^{s}_{i},i=1,2,3,4\}$. After that, We stack three SEA modules in the edge detection branch to aggregate the multi-level edge features $F_{e}$ in a coarse-to-fine manner and three EIA modules in the segmentation branch to aggregate segmentation features $F_{s}$. For clearer descriptions, We define the output of SEA modules as $E=\{E_{i},i=1,2,3\}$, $f^{e}_{4}$ as $E_{4}$, the output of EIA modules as $S=\{S_{i},i=1,2,3\}$ and $f^{s}_{4}$ as $S_{4}$. Multiple SEA modules and EIA modules are interacted in a cascaded manner to implement the cross refinement between the two branches. Moreover, in both two branches, the shallow detail features $f^{s}_{1}$ and $f^{e}_{1}$ are firstly fed into three cascaded GCA modules to filter the abundant background noise by leveraging the semantic stream from corresponding aggregation modules. We take the last EIA module’s prediction as the final segmentation result for testing stage.

The EIA module aims to inject the edge cues into the representation learning of segmentation features, and further explore the integrity information from both channel and spatial dimensions. Figure 3 shows the detail structure of EIA module, which consists of two stages, the first for feature fusion and the second for the integrity exploration.

Specifically, for the ith EIA module, given the segmentation feature $f^{s}_{i}\in{R^{{W_{1}}\times{H_{1}}\times 64}}$, the higher level segmentation feature $S_{i+1}\in{R^{{W_{2}}\times{H_{2}}\times 64}}$ from the previous EIA module and the edge feature $E_{i}\in{R^{{W_{1}}\times{H_{1}}\times 64}}$ from the previous SEA module as inputs. Inspired by [18], we first employ a mirror multiplication strategy consisting of two paths to fully aggregate the two-level segmentation features $f^{s}_{i}$ and $S_{i+1}$ (after up-sampling). In the main path, the higher level feature $S_{i+1}$ is fed into a $3\times 3$ convolutional layer to generate a semantic mask. This mask is then multiplied with $f^{s}_{i}$ to enhance the response of the camouflaged object. The mirror path, on the other hand, utilizes the lower level feature $f^{s}_{i}$ to generate a detail mask. This detail mask is then multiplied with $S_{i+1}$ to preserve fine-grained details that might have been lost in the coarser segmentation map. The two paths are concatenated with the edge feature $E_{i}$ and fed into a fusion block containing two $3\times 3$ convolutional layers to implement the guidance from the edge branch to the segmentation branch. Note that, the $3\times 3$ convolutional layer in this paper consists of a $3\times 3$ convolution, a batch normalization and a relu function. We denote the the final fused feature as $\widetilde{f^{s}_{i}}$, and the whole fusion process can be formulated as:

The SEA module aims to inject the integrity cues into the representation learning of edge features, which shares the same design ethos as the EIA module, but is much lighter.

Specifically, for the ith SEA module, given the edge feature $f^{e}_{i}\in{R^{{W_{1}}\times{H_{1}}\times 64}}$, the higher level edge feature $E_{i+1}\in{R^{{W_{2}}\times{H_{2}}\times 64}}$ from the previous SEA module, the segmentation feature $S_{i+1}\in{R^{{W_{2}}\times{H_{2}}\times 64}}$ from the EIA module as inputs. The two-level edge features $f^{e}_{i}$ and $E_{i+1}$ (after up-sampling) are concatenated and fed into a fusion block containing two $3\times 3$ convolutional layers for fully fusion, and then we adopt the mirror multiplication strategy $\mathcal{M}$ followed by an additional residual connection to inject the integrity cues from segmentation feature $S_{i+1}$ (after up-sampling) into the fused edge feature $f^{e}_{i}$. We denote the the final fused feature as $\widetilde{f^{e}_{i}}$, the whole fusion process can be formulated as:

In natural images, camouflaged objects tend to show smaller dimensions, which makes the structure details preserved in low-level features essential for detecting the integrity of camouflaged objects. Therefore, we propose the Guided-Residual Channel Attention (GCA) module and its structure is illustrated in Figure 2. The GCA module first applies a guide-flow block that guides semantic features from SEAs (EIAs in segmentation branch) to help filtering the background noise in the detail feature, then uses a channel attention block to further capture the inter dependencies between channels of the detail feature.

As shown in Figure 4, the guide-flow block takes detail features $D\in{R^{W_{1}\times H_{1}\times 64}}$ and semantic features $S\in{R^{W_{2}{\times}H_{2}{\times}64}}$ as inputs. The guidance map $G\in{R^{W_{1}\times H_{1}\times 1}}$ is obtained as:

where $C_{1\times 1}$ denotes the normalized $1\times 1$ convolutional layers. The guidance maps generated by both $S$ and $D$ can fully utilize the positional information of camouflaged objects in the semantic feature $S$ , while also taking into account the structural information of the camouflaged objects preserved in the detail feature $D$. The new detail feature $\widetilde{D}\in{R^{W_{1}\times H_{1}\times C}}$ is calculated as:

where $\cdot$ denotes the element-wise product. $C_{1\times 1}$ denotes the $1\times 1$ convolutional layer, then we fed it into a convolutional block followed by a multi-scale channel attention module $\textit{}{MS}$ to further explore its foreground representation in the channel dimension. Moreover, an additional residual connection is utilized to preserve the low-frequency structure information of camouflaged objects to the maximum extent possible. The process can be formulated as:

We utilise the co-supervision strategy to jointly train the two branchs. Following the previous state-of-the-art COD methods [1, 9], we adopt the weighted binary cross-entropy loss ($L^{w}_{BCE}$) and the weighted intersection-over-union loss ($L^{w}_{IOU}$) for the supervision of camouflaged mask ($M_{c}$), the dice loss ($L_{dice}$) for the supervision of edge mask ($M_{e}$). The total loss of EAMNet can be calculated as: $L_{total}=\sum_{i=1}^{3}{\lambda}_{i}(L^{w}_{BCE}(M_{c},P^{s}_{i})+L^{w}_{IOU}(M_{c},P^{s}_{i})+L_{dice}(M_{e},P^{e}_{i}))$, where ${\lambda}_{i}$ is a trade-off parameter and we set ${\lambda}_{i}=\frac{1}{2^{i-1}}$, $P^{s}_{i}$ is the segmentation prediction and $P^{e}_{i}$ is the edge prediction of camouflaged object.

Table I shows the quantitative comparison between EAMNet and 9 state-of-the-art methods in terms of $S_{\alpha}$, $E_{\phi}$, $F^{w}_{\beta}$ and $M$, and it can be observed that our EAMNet performs better than these methods in terms of all metrics. Specifically, in CAMO, our EAMNet outperforms the two edge-aware methods BGNet [9] and BSANet [10] by 3.5% and 1.9% in terms of structural similarity measure $S_{\alpha}$, respectively. This suggests that our EAMNet excels in mining the complete structure of the camouflaged object. Our improved performance can be attributed to the unique two-branch architecture, which allows for more accurate edge prior mining, and the EIA module, which can fully aggregate edge prior and segmentation features in both spatial and channel dimensions. Additionally, the proposed GCA module can make full use of the low-level features to boost COD performance.

We further evaluated the generalization ability of our EAMNet by testing it on the NC4K dataset, where it outperformed the second best method, BGNet, by 1.1%, 0.9%, and 1.3% in terms of $S_{\alpha}$, $E_{\phi}$, and $F^{w}_{\beta}$, respectively. Figure 5 provides visual comparisons between our EAMNet and the other methods in challenging scenes. Our method demonstrates superior performance in accurately detecting and segmenting the whole shape of the camouflaged object, as shown in the example of the Katydid image in the first row. Our EAMNet achieves the structural similarity measure $S_{\alpha}$ of 85%, while BGNet achieves only 60%.

To verify the effectiveness of the overall cross refinement strategy, we replace the whole edge detection branch with the boundary detection module in BSANet [10], which generates edge features only in the early stage using the multi-level features extracted by Res2Net. As shown in Table II, compared to model d, model e achieves obvious performance improvements on all benchmark datasets, with the performance gains of $1.3\%$, $0.9\%$, $1.7\%$ on average in terms of $S_{\alpha}$, $E_{\phi}$ and $F^{w}_{\beta}$. This is due to the lack of integrity information in the edge prior generated only early, and the cooperation between EIA module and SEA module in our EAMNet is beneficial to solve this problem. In addition, to verify the effectiveness of our GCA module, we remove the three GCA modules in the segmentation branch (sGCA) and the edge detection branch (eGCA), respectively. It can be observed that compared to module e, module c shows a significant decrease in all metrics, which indicates that the proposed GCA module’s powerful extraction of structural features in low-level features.