Mamba Capsule Routing Towards Part-Whole Relational Camouflaged Object Detection

The task of COD refers to accurately segmenting objects that are intentionally designed or naturally evolved to blend into their surroundings, which is rather challenging due to the high similarity between target object and background. Researchers have done a lot of wonderful works that have greatly advanced the development. In the early stage, most of the works are developed based on the hand-crafted features, e.g., colour Huerta et al (2007), 3D convexity Pan et al (2011) and intensity features Sengottuvelan et al (2008). However, they are relatively less robust and prone to fail in complex scenarios with low contrast situations. With the popularity of deep learning Fan et al (2020), recent works focus on mining more detailed features in a learning manner to distinguish camouflaged objects from their surroundings. Inspired by the biological mechanisms in nature or human visual psychological patterns, Fan et al. Fan et al (2022) mimiced the behavior process of predators to simulate the search and identification towards preys. Pang et al. Pang et al (2022) adopted the zoom mechanism of humans when they observed fuzzy objects for the task of COD. Jia et al. Jia et al (2022) segmented, magnified and reiterated the camouflaged object in a coarse-to-fine manner with the multi-stage strategy. Besides, some feature mining modules are elaborated for unearthing the subtle discriminative features of camouflaged objects. For example, Zhu et al. Zhu et al (2021) focused on texture-aware learning. Mei et al. Mei et al (2021) aimed to learn contextual-aware information. He et al. He et al (2023) and Zhong et al. Zhong et al (2022) introduced frequency clues to aid camouflaged object detection. Moreover, incorporating auxiliary tasks with the COD task can facilitate the precise segmentation map, such as classification Le et al (2019), edge/boundary detection Sun et al (2022); Zhu et al (2022); Luo et al (2024) and object ranking Lv et al (2021). The methods above, which are based on CNNs and Transformer, exhibit suboptimal performance in detecting camouflaging objects with high similarity and low contrast to their background. Alternatively, Liu et al. Liu et al (2021b) made the first attempt to complete COD task in the part-whole relational perspective successfully.

In this paper, in along with the pipeline of the part-whole relational COD, our work takes a further step in terms of the lightweight capsule routing, which is achieved by introducing VMamba Liu et al (2024c) to CapsNets Hinton et al (2018).

The history of part-whole representation goes back several decades. Krivic and Solina et al. Krivic and Solina (2004) recognized articulated objects based on part-level descriptions obtained by the Segmentor system Chen and Schmitt (1993). Girshick et al. Girshick et al (2015) designed a CNN to formulate the deformable part model using a distance transform pooling, object geometry filters, and maxout units. To address the problem of CNNs with space invariance, Hinton et al. Hinton et al (2011); Sabour et al (2017); Hinton et al (2018) explored the part-whole relationships by the CapsNets, which route low-level capsules (parts) to their familiar high-level ones (wholes). As the previous EM routing Hinton et al (2018) consumes too much computational resources, lots of improved works focus on the lightweight routing. Liu et al. Liu et al (2022b) disentangled two orthogonal 1D routings, which greatly reduce parameters and routing complexity, resulting in faster inference than the previous omnidirectional 2D routing adopted by the EM routing strategy. Liu et al. Liu et al (2024b) presented a residual pose routing. Likewise, Geng et al. Geng et al (2024) designed an orthogonal sparse attention routing to reduce redundancy and reducing parameters.

Despite the above improvement for capsule networks has made great progress in reducing the redundancy, the EM routing Hinton et al (2018) still remains the pixel level, resulting in large-scale capsule assignments and computational complexity. Unlike previous works, we introduce the VMamba Liu et al (2024c) to generate type-level mamba capsules from the pixel-level capsules for routing, which ensures a lightweight computation.

Considering that the high-order complexity of the self-attention mechanism in the transformer increases quadratically with increasing image size, mamba Gu and Dao (2024) has recently shown good performance in long sequence modeling and can be a promising alternative. Due to the low complexity, mamba has been involved in the computer vision community. For example, Zhu et al. Zhu et al (2024) designed the first mamba-based backbone network to generate a linear computational complexity while retaining advantages of vision transformer, which showcases superior performance and the ability to capture complex visual dynamics. Liu et al. Liu et al (2024c) designed a cross-scan mechanism to bridge the gap between 1D array scanning and 2D plain traversing. Ma et al. Ma et al (2024) proposed a hybrid CNN-SSM structure to capture local fine-grained features and remote dependencies in images to solve the problem of biomedical image segmentation. Liang et al. Liang et al (2024) introduced a reordering strategy to scan data in a specific sequence to capture point cloud structures.

In this paper, inspired by the computational efficiency, we introduce VMamba Liu et al (2024c) in CapsNets Hinton et al (2018) to solve the part-whole relational COD task, which generates the type-level mamba capsules from the pixel-level versions using the implicit hidden state for further capsules routing.

Fig. 2 depicts the overall architecture of our proposed MCRNet, including a Swin Transformer encoder Liu et al (2021c), a Mamba Capsule Generation (MCG) module, a Capsules Spatial Details Retrieval (CSDR) module, and a multi-task learning decoder Liu et al (2021a). To be specific, the input image is divided into non-overlapped patches after data augmentation, which are fed into Swin Transformer Liu et al (2021c) to capture long-range context features. On top of that, MCG is designed to generate type-level mamba capsules from the pixel-level capsules for further capsule routing to obtain the high-level mamba capsules. To learn the spatial details of mamba capsules, CSDR is developed to retrieve the spatial resolution of the high-layer type-level mamba capsules. Finally, a multi-task learning decoder including camouflaged detection and edge detection is designed to detect the camouflaged object with excellent boundary.

The transformer encoder first splits the input RGB image $\mathbf{I}\in\mathbb{R}^{C\times H\times W}$ into non-overlapped patches ($p\times p$) by a patch embedding module, where $C$, $H$ and $W$ denote channel size, height and width of image $\mathbf{I}$, respectively, and $p$ = 16. These image patches are linearly projected into a 1D sequence of token embeddings $\mathbf{F}^{E}\in\mathbb{R}^{l\times d}$, where $l=HW/p^{2}$ and $d$ are the length of the patch sequence and the channel dimension, respectively. The Swin Transformer Liu et al (2021c) encoder is used to capture global dependencies $\mathbf{F}_{i}^{E}\in\mathbb{R}^{l_{i}\times d_{i}}$, where $i\in[0,1,2,3]$ indicates the index of blocks in the encoder, $l_{i}$ and $d_{i}$ mean the length of the sequence and the channel dimension of the token, respectively. Its unique shifted windowing mechanism reduces the computational burden with more efficient batch computation, showing efficiency and high performance.

In this subsection, we will detail the MCG module that learns the type-level mamba capsules from the pixel-level capsules, which is composed by primary capsules generation, multi-direction serialization, implicit latent state learning and mamba capsule acquisition.

Step 1: Primary capsules generation. As shown in Fig. 3, the feature sequence $\mathbf{F}_{2}^{E}\in\mathbb{R}^{l_{2}\times d_{2}}$ obtained by the encoder is reshaped into $\mathbf{F}^{{}^{\prime}}\in\mathbb{R}^{h_{2}\times w_{2}\times d_{2}}$ to facilitate subsequent Primary Capsules (PrimaryCaps) generation $\mathbf{P}\in\mathbb{R}^{h_{2}\times w_{2}\times{O}\times U}$, which contains the pose matrix $\mathbf{P}_{pose}\in\mathbb{R}^{h_{2}\times w_{2}\times{O}_{P}\times U}$ and the activation value $\mathbf{P}_{act}\in\mathbb{R}^{h_{2}\times w_{2}\times{O}_{A}\times U}$, where $O=\{{O}_{P}=16,{O}_{A}=1\}$ is the dimension of the pose matrix and the activation, $U$ represents the number of primary capsules, i.e.,

where $\Phi(\cdot)$ represents the operation of convolution, batch normalization and relu. $\mathrm{Sigmoid}(\cdot)$ means the sigmoid function. $\mathrm{Cat}(\cdot)$ represents the concatenation.

Step 2: Multi-direction serialization. Using the scanning mechanism of VMamba Liu et al (2024c) for providing more accurate and rich 2D spatial context, the 2D primary capsules are serialized to four groups of 1D capsule sequences $\mathbf{S}_{g}=\{\mathbf{S}_{1},\dots,\mathbf{S}_{G}\}\in\mathbb{R}^{V\times O\times U}$, where $G=4$ means four various scanning directions as shown in Fig. 3 (positive Z shape, inverted Z shape, positive N shape, and inverted N shape), $U$ represents the number of capsule sequences, $V=h_{2}\times w_{2}$ is the length of the capsule sequence and $O$ means the dimension of the capsule token. In a certain scanning direction $g$, each capsule sequence $\mathbf{S}_{g}(u)=\{\mathbf{S}_{g}(1),\dots,\mathbf{S}_{g}(U)\}\in\mathbb{R}^{V\times O}$ represents various part object. In a certain capsule sequence $\mathbf{S}_{g}(u)$, there are $V$ capsule tokens $\mathbf{S}_{g}(u,v)=\{\mathbf{S}_{g}(u,1),\dots,\mathbf{S}_{g}(u,V)\}\in\mathbb{R}^{O}$.

Step 3: Implicit latent state learning. During the selective SSM, the current latest implicit latent state is associated with both the accumulated latent state and the current input token, which can be formulated as

where $\mathbf{S}_{g}(u,v)$ represents the ${v}^{th}$ token in the ${u}^{th}$ capsule sequence obtained in the scanning direction $g$. $\mathbf{h}_{g}(u,v)\in\mathbb{R}^{N}$ means the updated implicit latent state after the token $\mathbf{S}_{g}(u,v)$ is input. $\overline{\mathbf{A}}$ and $\overline{\mathbf{B}}$ represent the discretized evolution parameters of the model, which will be computed based on the input sequence.

The most recent output mamba token can be obtained by utilizing this latest latent state

where $\mathbf{F}_{g}^{M}(u,v)$ indicates the output mamba token corresponding to $\mathbf{S}_{g}(u,v)$. $\mathbf{C}$ represents the projection parameter of the model, which is computed relying on the input sequence.

As each token $\mathbf{S}_{g}(u,v)$ in the sequence $\mathbf{S}_{g}(u)$ is fed into the selective SSM, the implicit latent state $\mathbf{h}_{g}(u,v)$ is updated constantly. The final implicit latent state $\mathbf{h}_{g}(u,V)$, which is defined as mamba capsule vectors completes the global modeling of the sequence $\mathbf{S}_{g}(u)$. Algorithm 1 lists the process of mamba capsule vectors learning in details.

Step 4: Mamba capsule acquisition. Ultimately, in the certain scanning direction $g$, we obtain learned mamba sequences $\mathbf{F}_{g}^{M}\in\mathbb{R}^{V\times O\times U}$, and the implicit latent state $\mathbf{h}_{g}\in\mathbb{R}^{N\times U}$. Due to the fact that the final latent state $\mathbf{h}_{g}(u,V)$ implicitly explores the global context of the corresponding sequence, we choose it as the capsule pose vector $\mathbf{M}_{pose,g}\in\mathbb{R}^{N\times U}$, which has a comprehensive understanding about the pixel-level capsules. Based on the pose vector $\mathbf{M}_{pose,g}$, we can compute the activation values $\mathbf{M}_{act,g}\in\mathbb{R}^{O_{A}\times U}$ through

To this end, the type-level Mamba Capsules (MambaCaps) $\mathbf{M}_{g}\in\mathbb{R}^{1\times 1\times O\times U}$ is constructed as

where $\mathrm{Unsqueeze}(\cdot)$ represents the operation of unsqueeze. In Eq. (10), the type-level mamba capsules are derived from the pixel-level capsules while preserving global context, which helps to get routing computation reduced significantly.

In this subsection, we will detail the CSDR module for further camouflaged prediction, which is composed by high-layer mamba capsules learning, adjacent-layer mamba capsules correlation and high-layer capsules spatial details retrieval.

Step 1: High-layer mamba capsules learning. As shown in Fig. 4, under the certain scanning direction $g$, the obtained mamba capsules are fed into EM routing Hinton et al (2018) with iterative refinement for mining the part-whole relationship at the type-level, and the high-layer mamba capsules can be computed via

where $\mathrm{EM}(\cdot)$ represents the EM routing algorithm Hinton et al (2018) and $\widetilde{\mathbf{M}}_{g}=[\widetilde{\mathbf{M}}_{g}(1),\ldots,\widetilde{\mathbf{M}}_{g}(U)]\in\mathbb{R}^{1\times 1\times O\times U}$ means $U$ explored high-layer mamba capsules under the certain scan direction $g$.

Step 2: Adjacent-layer mamba capsules correlation. To utilize the type-level mamba capsules for the dense prediction of camouflaged object, it is necessary to retrieve the spatial details of the mamba capsules.

Under the certain scanning direction $g$, the cosine similarity matrix $\mathbf{E}_{g}(m,n)\in\mathbb{R}^{U\times U}$ depicts the similarity degree between the adjacent-layer type-level mamba capsules $\mathbf{M}_{g}(m)$ and $\widetilde{\mathbf{M}}_{g}(n)$, where $m\in[1,\dots,{U}]$ is the row index of the matrix $\mathbf{E}_{g}$ and the index of $\mathbf{M}_{g}$, $n\in[1,\dots,{U}]$ represents the column index of the matrix $\mathbf{E}_{g}$ and the index of $\widetilde{\mathbf{M}}_{g}$, which can be computed by

where $x\in[1,\dots,O]$, $y\in[1,\dots,O]$ represent the element index of $\mathbf{M}_{g}(m)$ and $\widetilde{\mathbf{M}}_{g}(n)$, respectively.

To further enhance the differentiation and distinguishable ability of the cosine similarity between adjacent-layer mamba capsules, the sigmoid function is utilized for activation as

In the cosine similarity matrix $\mathbf{\widehat{E}}_{g}$, the element in row $m$ and column $n$ represents the degree of correlation between the ${m}^{th}$ low-layer mamba capsule and the ${n}^{th}$ high-layer mamba capsule, which also reflects the correlation between the corresponding pixel level adjacent layer capsules.

Step 3: High-layer capsules spatial details retrieval. Under the certain scanning direction $g$, the activation values $\mathbf{S}_{act,g}\in\mathbb{R}^{V\times O_{A}\times U}$ of the capsule sequences is multiplied with the activated cosine similarity matrix to obtain the feature map $\mathbf{F}_{g}^{R}\in\mathbb{R}^{V\times O_{A}\times U}$ i.e.,

where $\mathrm{Split}(\cdot)$ represents the operation of split the final dimension along the channel axis.

The obtained relational sequences $\mathbf{F}_{g}^{R}$ is integrated with learned mamba sequences $\mathbf{F}_{g}^{M}$ to obtain the sequence $\mathbf{F}_{g}^{C}$. Finally, four sequence $\mathbf{F}_{g}^{C}$ are integrated under multiple scanning directions through flip and transpose operations to ensure consistency with the first scanning direction

where $\mathrm{MSA}(\cdot)$ means multi-head self-attention. $\Psi(\cdot)$ and $\Gamma(\cdot)$ represent the operations of transpose and flipping, respectively.

In the decoder, following the idea of VST Liu et al (2021a), two designed task-related tokens (i.e., a camouflage token $\mathbf{t}^{c}\in\mathbb{R}^{1\times d}$ and a boundary token $\mathbf{t}^{b}\in\mathbb{R}^{1\times d}$) are added on the obtained tokens $\mathbf{F}^{D}$ for completing camouflaged object segmentation and edge detection distinctively. Then, the all tokens are processed via transformer layers to capture global dependencies. In every layer, a patch-task-attention between $\mathbf{F}_{j}^{D}$ and $\mathbf{t}_{j}^{c}$ is designed for camouflage prediction $\mathbf{F}_{j}^{c}$, where $j\in[0,1,2]$ indicates the index of blocks in the decoder. $\mathbf{F}_{j}^{D}$ is mapped to queries $\mathbf{Q}_{j}^{c}\in\mathbb{R}^{l_{j}\times d_{j}}$ and $\mathbf{t}_{j}^{c}$ is mapped to a key $\mathbf{k}_{j}^{c}\in\mathbb{R}^{1\times d_{j}}$ and a value $\mathbf{v}_{i}^{c}\in\mathbb{R}^{1\times d_{j}}$, where $l_{j}$ and $d_{j}$ mean the length of the sequence and the channel dimension of the token, respectively. The camouflage prediction $\mathbf{F}_{j}^{c}$ can be computed by

where $(\cdot)^{T}$ and $\oplus$ represent the transpose operation of the matrix and the operation of element-wise addition, respectively. In a similar way, for boundary prediction, $\mathbf{F}_{j}^{D}$ is mapped to queries $\mathbf{Q}_{j}^{b}\$and $\mathbf{t}_{j}^{b}$ is mapped to a key $\mathbf{k}_{j}^{b}$ and a value $\mathbf{v}_{j}^{b}$ to gain the result

Whereafter, two linear transformations are applied with the sigmoid activation to project $\mathbf{F}_{j}^{c}$, $\mathbf{F}_{j}^{b}$ to scalars in [0, 1]. Therefore, get the final 2D camouflaged map $\mathbf{\widetilde{F}}_{j}^{c}$ and boundary map $\mathbf{\widetilde{F}}_{j}^{b}$ at the corresponding scale.

In this work, both the weighted binary cross-entropy (BCE) loss function ($l_{wbce}$) and the Intersection over Union (IoU) loss ($l_{iou}$) Yu et al (2016) are adopted as loss functions to train the network. Suppose $\mathbf{\widetilde{F}}^{c}$, $\mathbf{\widetilde{F}}^{b}$, $\mathbf{G}^{c}$ and $\mathbf{G}^{b}$ are the predicted camouflaged map, boundary map, corresponding camouflaged and boundary ground truth, respectively. $l_{wbce}$ can be expressed in the formula as follows:

where $j$ is the index of blocks in the decoder and $w_{j}$ is a set of hyperparameters that we set the value of $[w_{0},w_{1},w_{2},w_{3}]$ to $[1,\ 0.8,\ 0.5,\ 0.5]$. In Eq. (18)

where $m$ represents the pixel index and $n$ means the total number of pixels.

$l_{iou}$ is defined on the input scale,

Dataset. We evaluate the proposed method on three widely public benchmarks.

CAMO Le et al (2019) is the first COD dataset, containing 1,250 camouflaged images with 1,000 training images and 250 testing images.

COD10K Fan et al (2020) is a currently large COD datasets, consisting of 3,040 training images and 2,026 testing images..

NC4K Lv et al (2021) is a recently released large-scale COD dataset containing 4,121 images.

To ensure consistency with previous studies Fan et al (2022), 3040 samples from COD10K and 1000 samples from CAMO are utilized as the training set, while the test set consisting of 2026 test images in COD10K, 250 test images in CAMO and the entire NC4K dataset.

Evaluation metrics. Four commonly-used metrics are employed for COD task to assess model performance, including Mean Absolute Error $(MAE)$ Achanta et al (2009), maximum F-measure $F_{m}$ Margolin et al (2014), maximum enhanced-alignment measure $E_{m}$ Fan et al (2018) and structure-measure $S_{m}$ Fan et al (2017). Given a continuous camouflaged map, a binary mask $\hat{F}$ is achieved by thresholding the camouflaged map $F$. Precision is defined as $Precision={{\left|{\hat{F}\cap G}\right|}\mathord{\left/{\vphantom{{\left|{\hat{F}\cap G}\right|}{\left|\hat{F}\right|}}}\right.\kern-1.2pt}{\left|\hat{F}\right|}}$, and recall is defined as $Recall={{\left|{\hat{F}\cap G}\right|}\mathord{\left/{\vphantom{{\left|{\hat{F}\cap G}\right|}{\left|G\right|}}}\right.\kern-1.2pt}{\left|G\right|}}$.

$MAE$ is defined as

where $\hat{W}$ and $\hat{H}$ are the width and height of the image, respectively.

Maximum F-measure ($F_{m}$) is the maximum value of the F-measure $(F_{\beta})$ under different thresholds. F-measure $(F_{\beta})$ is an overall performance indicator, which is computed by

As suggested in Margolin et al (2014), ${{\beta^{2}}=0.3}$.

Maximum enhanced-alignment measure ($E_{m}$) is the maximum value of E-measure under different thresholds, which combines local pixel values with the image-level mean value to jointly evaluate the similarity between the prediction and the ground truth.

Structure-measure ($S_{m}$) is computed by

where $S_{o}$ and $S_{r}$ represent the object-aware and region-aware structure similarities between the prediction and the ground truth, respectively. $\alpha$ is set to 0.5 Fan et al (2017).

Implementation details. The proposed MCRNet is implemented by PyTorch. The tiny Swin-transformer Liu et al (2021c) is adopted as the network encoder. Other modules are randomly initialized. Each image is resized to 384×384 pixels and then randomly cropped to 352×352 for training. The network employs the Adam optimizer Kingma and Ba (2015) with an initial learning rate of 0.0001, which is reduced by a factor of 10 at half and three-quarters of the total training steps. The complete training process contains a total of 150,000 training steps with a batch size of 8 using a 4090 GPU.

In order to demonstrate the efficacy of the proposed method, the comparative analysis is conducted with 25 recent state-of-the-art methodologies, including the capsule network based method (i.e., POCINet Liu et al (2021b)), the CNNs based methods ( SINet Fan et al (2020), MGL Zhai et al (2021), PFNet Mei et al (2021), LSR Lv et al (2021), C2FNet Sun et al (2021), UJSC Li et al (2021), R-MGL-v2 Zhai et al (2023), SLTNet Cheng et al (2022), OCENet Liu et al (2022a), BSANet Zhu et al (2022), FAPNet Zhou et al (2022), BGNet Sun et al (2022), SegMaR Jia et al (2022), SINet-v2 Fan et al (2022), FDCOD Zhong et al (2022), ZoomNet Pang et al (2022), PopNet Wu et al (2023), FEDER He et al (2023), DGNet Ji et al (2023), DINetZhou et al (2024)) and the transformer based methods (UGTR Yang et al (2021), OSFormer Pei et al (2022), FDCOD Zhong et al (2022), VSCode Luo et al (2024)). For a fair comparison, all the predictions of these methods are either provided by the authors or generated by models retrained based on the open source codes with the same code.

Quantitative analysis. Table 1 presents a summary of the quantitative analysis of the proposed approach in contrast to 25 rivals on three COD datasets in terms of four evaluation metrics. From the metric data, it can be seen that the proposed MCRNet comprehensively surpasses all existing state-of-the-art methods. Compared to the second best and transformer based network called VSCode Luo et al (2024), the method achieves average performance gains of 8.5%, 1.7%, 0.2%, 1.0% in terms of $MAE$, $F_{m}$, $E_{m}$, $S_{m}$, respectively after averaging all metrics of these three datasets. Compared with FEDER He et al (2023) based on CNN, which also completes object segmentation and edge detection tasks, it shows significant performance improvements of 21.1%, 5.1%, 2.9%, and 4.5% respectively in the four indicators from the average perspective. Compared to the multi-scale method ZoomNet Pang et al (2022), the average gains are 15.9%, 4.1%, 2.0%, and 2.8%, respectively. Besides, compared with the CapsNets based method POCINet Liu et al (2021b), we have achieved a significant improvement in segmentation accuracy, which benefits from the mamba capsule routing in our MCRNet.

Qualitative analysis. Fig. 5 presents the segmentation visualizations of the MCRNet with ten good methods. From the three test sets, camouflage objects of diverse sizes and camouflage scenes of various types are selected for visualizations, encompassing small objects, large objects, the objects with uncertain boundaries, the objects that are obscured, and the concealed persons. As can be witnessed from the first and second rows, the small camouflage objects can be detected extremely well and not missed, particularly the second row of camouflage objects with a high resemblance to the background. In the instance of large camouflaged objects, they can be identified with good completeness. In the case of fuzzy boundaries, the camouflaged object can be detected entirely from the low-contrast background. It is worthy of mention that for objects obscured by the jungle, they can be distinguished meticulously. Similarly a favorable detection effect has also been attained in the scene containing persons. The aforementioned outstanding segmentation of camouflaged objects are attributed to the exploration of the part-whole relationship by the proposed MCRNet.

Effectiveness of MCG and CSDR. The proposed MCG and CSDR module exert a significant role in facilitating proposed MCRNet for part-whole relational camouflaged object detection. To dig into the contributions of these two components, we design ablation studies by removing them from the entire framework. Table 2 and Fig. 6 demonstrate the performance and visualizations for the ablation study. Comparing the fourth and fifth rows in Fig. 6, it can be observed that MCG enables to better separate the camouflage object from its surroundings, which is also proven in Table 2 (a) and (b) in terms of performance. This is attributed to the mamba capsules by the latent state mechanism in the selective SSM Gu and Dao (2024) model that realizes the modeling of global spatial structure information. Comparing the third and fourth rows in Fig. 6, it can be seen that our CSDR can effectively detect what other models cannot and enhance the integrity of camouflaged object. Similar conclusion can be achieved by comparing Table 2 (b) and (c). This proves that the spatial details retrieved by CSDR help the segmentation of the camouflaged objects.

Generation of type-level capsules. To prove the validity of MCG module for type-level mamba capsules generation, we compare it with a straightforward manner that uses a linear mapping to generate the type-level capsules. As shown in Table 3, capsules generated with mamba outperform those generated with the fully-connected layer. This superiority can be attributed to that the latent state using the selective SSM Gu and Dao (2024) in VMamba Liu et al (2024c) enables accumulates selected token information for comprehensive global context. By contrast, full connection simply implements a weighted sum of all tokens in the sequence without retaining spatial structure context well.

Number of mamba capsules. Table 4 demonstrates the impact of the quantity of mamba capsules in the proposed network on the model’s detection capability. As is widely acknowledged in Table 4, insufficient capsules undermine the characterization ability of camouflaged objects, while excessive capsules result in overfitting and a decline in detection performance. After conducting a qualitative analysis the optimal equilibrium in terms of detection performance and generalization ability is to set to 32 for mamba capsules number, which is the setting in this paper to facilitate the efficient experiments.

Scanning direction for capsules sequence. Table 5 explores various serialization directions to study the scanning directions in the MCRNet, including one direction, two directions and four directions, which can be referred to Fig. 3. Specifically, one direction and two directions possess the scannings of ’Z’ and ’Z’ & ’N’, respectively. In Table 5, the combination of four scannings achieves the best performance, which indicates the capability of various scanning directions for global context extraction.

Efficiency analysis. To explore the efficiency of the proposed MCRNet for the pipeline of part-whole relational COD based on CapsNets, we replace the proposed mamba capsule routing with some previous capsule routing algorithms, including EM routing Hinton et al (2018) and Disentangled Capsule Routing (DCR) Liu et al (2022b) in the entire MCRNet framework. In Table 6, the proposed mamba capsule routing achieves the lowest FLOPs, and parameters, and highest inference speed, while performing best on various datasets, which demonstrates the complexity efficiency and performance superiority.

Fig. 7 displays some failure cases of our camouflaged detector on extremely complex scenes. For instance, as depicted in the left two columns of Fig. 7, the model’s detection of camouflaged targets is influenced by salient target detection, leading to unnecessary object parts being detected and a shift in detection focus. Besides, in the scene shown in the right two columns of Fig. 7, subpar performance exhibits in detecting small objects out of the center due to increased observation angle distance, making it challenging to discern camouflage on small objects. In future endeavors, on the basis of leveraging part-whole relational methods, we will leverage the power of Large Language Model (LLM) Ouyang et al (2022) to help the understanding of the camouflaged scene for better concealed object searching and identification.