Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification

Inspired by the advances in feature pruning (Liang et al. 2022) and person ReID (He et al. 2021), we use a transformer with token sparsification to prune interference from occlusion and background. As shown in fig. 2, given an input image $x\in\mathbb{R}^{H\times W\times C}$, where $W,H,C$ denote the width, height, and channel of the image respectively, we split $x$ into N overlapping patches $\{p_{1},p_{2},\cdots,p_{N}\}$ and embed each patch with linear projection denoted as $f(\cdot)$. Then we combined a learnable $[\mathrm{cls}]$ token with patch embedding and apply positional encoding and camera index encoding following (He et al. 2021). The final input can be described as:

$$\mathcal{Z}=\left\{x_{cls},f\left(p_{1}\right),\cdots,f\left(p_{N}\right)\right\}+\mathcal{P}+\mathcal{C}_{id}$$

(1)

where $x_{cls}\in\mathbb{R}^{1\times D}$ is learnable $[\mathrm{cls}]$ token. $f(p_{i})\in\mathbb{R}^{1\times D}$ is i-th patch embeddings. $\mathcal{P}\in\mathbb{R}^{(N+1)\times D}$ is position embeddings and $\mathcal{C}_{id}\in\mathbb{R}^{(N+1)\times D}$ is camera index embeddings.

Token Sparsification. We adopt the token sparsification strategy proposed in (Liang et al. 2022). Specifically, through the attention correlation (Vaswani et al. 2017) between $[\mathrm{cls}]$ token and other tokens in the vision transformer, we can express the value of $[\mathrm{cls}]$ token as:

$$x_{cls}=A_{cls}V=\mathrm{softmax}(\frac{Q_{cls}K}{\sqrt{d}})V$$

(2)

where $A_{cls}$ denotes the attention matrix of the $[\mathrm{cls}]$ token, i.e., first row of the attention matrix, $\sqrt{d}$ is scale factor. $Q_{cls},K,V$ represent the query matrix of $[\mathrm{cls}]$ token, the key matrix, and the value matrix respectively. For multiple heads in the self-attention layer, we average the attention matrix as $\bar{A}_{cls}={\textstyle\sum_{i}^{n}\frac{1}{n}\cdot A_{cls}^{(i)}}$, n is the total heads number. Since the $[\mathrm{cls}]$ token corresponds to a larger attention value in significant patch regions (Caron et al. 2021), we can evaluate the importance of a token according to its relevance to the $[\mathrm{cls}]$ token. As $\bar{A_{cls}}$ represents the correlation between $[\mathrm{cls}]$ token and all other tokens, we hence preserve the tokens with $\mathcal{K}$ largest values in $\bar{A_{cls}}$ and drop others, which is shown in fig. 2. We define $\mathcal{K}=\left\lceil\gamma\cdot N_{c}\right\rceil$, where $\gamma$ is the keep rate and $N_{c}$ is the total number of tokens in the current layer, $\left\lceil\cdot\right\rceil$ is the ceiling operation. With token sparsification, the preserved tokens are mostly related to the region of the target pedestrian, and dropped tokens are related to occluders or backgrounds.

Sparse Encoder Supervision Loss. We use the cross-entropy ID loss and triplet loss to supervise $[\mathrm{cls}]$ token $x_{cls}^{\mathcal{S}}$ obtained by the sparse encoder as:

$$\mathcal{L}_{S}=\mathcal{L}_{ID}(x_{cls}^{\mathcal{S}})+\mathcal{L}_{T}(x_{cls}^{\mathcal{S}})$$

(3)

where $\mathcal{L}_{ID}$ denotes cross-entropy ID loss and $\mathcal{L}_{T}$ denotes triplet loss. Compared with existing approaches, our feature pruning with token sparsification is adaptive and doesn’t rely on prior knowledge of human semantics, and can better handle challenging scenarios, such as heavy occlusion and other persons as occluders, as shown in fig. 6.

After the feature pruning, we would like to find the most related patches from other views that not been occluded for consolidation, such strategy has been proved to be effective (Xu et al. 2022; Yu et al. 2021). Specifically, we first learn a gallery memory with the pre-trained encoder. With the pruned query image features, we rank patches in the gallery memory according to their similarity with the query. Considering that there exists appearance gap between the pruned query feature and holistic feature in gallery memory, we measure the similarity from both the image-level and patch-level. As shown in fig. 1, we linearly combine image-level and patch-level distance to match the query image with gallery memory images.

Image-level distance. We define the image-level distance as the cosine similarity between the $[\mathrm{cls}]$ tokens of query and gallery memory as follows:

$$\mathcal{D}_{COS}=1-\frac{\langle x_{cls}^{(i)},x_{cls}^{(i)}\rangle}{|x_{cls}^{(i)}|\cdot|x_{cls}^{(j)}|}$$

(4)

where $x_{cls}^{(i)}$ and $x_{cls}^{(j)}$ is $[\mathrm{cls}]$ token of i-th query image and j-th image in gallery memory. $\left\langle\ \cdot\ \right\rangle$ is the dot product.

Patch-level distance. We leverage Earth Mover’s Distance (EMD) (Rubner, Tomasi, and Guibas 2000) to measure patch-level similarity. EMD is usually employed for measuring the similarity between two multidimensional distributions and is formulated as following linear programming problem: Let $\mathcal{Q}=\left\{(q_{1},w_{q_{1}}),\dots,(q_{m},w_{q_{m}})\right\}$ be the set of patch tokens from the query image, where $q_{i}$ is i-th patch token and $w_{q_{i}}$ is the weight of $q_{i}$. Similarly, $\mathcal{G}=\left\{(g_{1},w_{g_{1}}),\dots,(g_{n},w_{g_{n}})\right\}$ represents set of patch tokens of a gallery image. Then $w_{q_{i}}$ is further defined as proportional correlation weight (Phan and Nguyen 2022) of $q_{i}$ to set $\{g_{1},g_{2},\dots,g_{n}\}$, denoted as $\max(0,\langle q_{i},\frac{1}{n}\sum_{i}^{n}g_{i}\rangle)$, and $w_{g_{j}}$ equals $\max(0,\langle g_{j},\frac{1}{m}\sum_{i}^{m}q_{i}\rangle)$. The ground distance $d_{ij}$ is the cosine distance between $p_{i}$ and $q_{j}$. The objective is to find the flow $\mathcal{F}$, of which $f_{ij}$ indicates the flow between $p_{i}$ and $q_{j}$, to minimize following cost:

$$\begin{split}\mathcal{F}^{*}&=\operatorname*{arg\,min}_{\mathcal{F}}\mathrm{COST}(\mathcal{Q},\mathcal{G},\mathcal{F})\\ &=\operatorname*{arg\,min}_{\mathcal{F}}\left(\sum_{i=1}^{m}\sum_{j=1}^{n}f_{ij}d_{ij}\right)\end{split}$$

(5)

subject to the following constraints:

$$f_{ij}\geq 0,\ 1\leq i\leq m,\ 1\leq j\leq n$$

(6)

$$\sum_{i=1}^{m}f_{ij}\leq w_{g_{j}},\ 1\leq j\leq n$$

(7)

$$\sum_{j=1}^{n}f_{ij}\leq w_{q_{i}},\ 1\leq i\leq m$$

(8)

$$\sum_{i=1}^{m}\sum_{j=1}^{n}f_{ij}=\min(\sum_{i=1}^{m}w_{q_{i}},\sum_{j=1}^{n}w_{g_{j}})$$

(9)

eq. 5 can by solved by iterative Sinkhorn algorithm (Cuturi 2013) and produces the earth mover’s distance $\mathcal{D}_{EMD}$. Intuitively, $\mathcal{D}_{COS}$ represents global distance, while $\mathcal{D}_{EMD}$ measures in the perspective of the set of local features. Naturally, we take the linear combination of image-level cosine distance and patch-level EMD as our final distance for feature matching, expressed as follows:

$$\mathcal{D}=(1-\alpha)\mathcal{D}_{COS}+\alpha\mathcal{D}_{EMD}$$

(10)

With $\mathcal{D}$, we rank the sparse query feature with holistic features in gallery memory and generate a ranking list. In order to expedite the process of feature matching, we use an efficient two-stage selection strategy to save time costs by 138 times.

After we retrieve the $K$ nearest gallery neighbors, we then consolidate the pruned query feature from multi-view observations in the gallery memory. Specifically, we average the $[\mathrm{cls}]$ tokens of the query and gallery neighbors as our initial global feature. Then we combine the averaged $[\mathrm{cls}]$ token with patch tokens in both query and gallery neighbors to aggregate information of multi-view pedestrians. The consolidation of the multi-view features can be formulated as follows:

$$f_{m}=\left\{\bar{f_{c}},f_{p_{q}},f_{p_{g}}^{{(1)}},f_{p_{g}}^{(2)},\dots,f_{p_{g}}^{(K)}\right\}$$

(11)

where $\bar{f_{c}}\in\mathbb{R}^{1\times C}$ is the average of $\mathrm{[cls]}$ tokens of both query and its neighbors. $f_{p_{q}}\in\mathbb{R}^{M\times C}$ is the patch tokens of query and $M$ is the number of patch tokens. $f^{(i)}_{p_{g}}\in\mathbb{R}^{N\times C}$ is the patch tokens of i-th gallery neighbor and $N$ is the number of patch tokens for each gallery.

Transformer Decoder. Notice that $[\mathrm{cls}]$ token is class prediction (Dosovitskiy et al. 2020) in transformer and conventionally treated as global feature for image representation, the query $\mathrm{[cls]}$ token is usually contaminated in the occluded person ReID problem . We send the consolidated multi-view feature to a transformer decoder to compensate the incomplete $\mathrm{[cls]}$ token with gallery neighbors. In the decoder, we first transform the multi-view feature $f_{m}$ into $Q,K,V$ vectors using linear projections, which is:

$$Q=W_{q}\cdot f_{m},K=W_{k}\cdot f_{m},V=W_{v}\cdot f_{m}$$

(12)

where $W_{q},W_{k},W_{v}$ are weights of linear projections. As the multi-view feature contains three parts: $\mathrm{[cls]}$ token, patch tokens of query, and patch tokens of gallery neighbors, the computation of attention with respect to $\mathrm{[cls]}$ token can be decomposed into the above three parts, which is expressed as:

$$\begin{split}x_{cls}&=\mathrm{Cat}(A^{{}^{\prime}}_{c},A^{{}^{\prime}}_{q},A^{{}^{\prime}}_{g})\cdot\mathrm{Cat}(V_{c},V_{q},V_{g})\\ &=A^{{}^{\prime}}_{c}\cdot V_{c}+A^{{}^{\prime}}_{q}\cdot V_{q}+A_{g}^{{}^{\prime}}\cdot V_{g}\end{split}$$

(13)

where $\mathrm{Cat}\left(\ \cdot\ \right)$ is operation of vector combination. $A^{{}^{\prime}}$ denotes the attention matrix of $\mathrm{[cls]}$ token and $V$ is the value vector. Subscripts $c,q,g$ correspond to $\mathrm{[cls]}$ token, query and gallery neighbors respectively. $A_{c}^{{}^{\prime}}\cdot V_{c}+A_{q}^{{}^{\prime}}\cdot V_{q}$ acts as the feature learning process with sparse query and $A_{g}^{{}^{\prime}}\cdot V_{g}$ integrates completion information from gallery neighbors to the $\mathrm{[cls]}$ token. The final consolidated $[\mathrm{cls}]$ token generated by the transformer decoder is denoted as $x_{cls}^{\mathcal{C}}=\tau(f_{m})$. We find one layer of transformer is enough and also avoids the high memory consumption from neighborhoods (Yu et al. 2021).

Consolidation Loss. The $[\mathrm{cls}]$ token $x_{cls}^{\mathcal{C}}$ obtained by feature consolidation decoder is supervised with cross entropy loss as ID loss and triplet loss, as expressed below:

$$\mathcal{L}_{C}=\mathcal{L}_{ID}(x_{cls}^{\mathcal{C}})+\mathcal{L}_{T}(x_{cls}^{\mathcal{C}})$$

(14)

Therefore, the final loss function can be expressed:

$$\mathcal{L}=\mathcal{L}_{S}+\mathcal{L}_{C}$$

(15)

We choose the ViT-B/16 as our backbone for both the sparse encoder and gallery encoder. Our sparse encoder incorporates several modifications into the backbone, including camera encoding, patch overlapping with a stride of 11, batch normalization, and token sparsification at layer 3, 6, 9. Our gallery encoder has the same structure as the sparse encoder but does not conduct token sparsification. We construct the gallery memory with gallery image tokens as in eq. 1. In the multi-view feature matching module, we first identify 100 globally-similar gallery neighbors using cos distance in eq. 4, and then search the final $K$ nearest gallery neighbors using the proposed distance in eq. 10. We set the number of $K$ to 10 on Occluded-Duke dataset and 5 for others. In the training process, we resize all input images to 256 $\times$ 128. The training images are augmented with random horizontal flipping, padding, random cropping and random erasing (Zhong et al. 2020). The batch size is 64 with 4 images per ID and the learning rate is initialized as 0.008 with cosine learning rate decay. The distance weight $\alpha$ in eq. 10 is 0.4 and the parameter of keep rate $\gamma$ is 0.8. We take consolidated $\mathrm{[cls]}$ token in eq. 13 for model inference. For large-scale problems, we can further replace the full image tokens with [cls] tokens to save computation and memory.

Occluded-Duke (Miao et al. 2019) includes 15,618 training images of 702 persons, 2,210 occluded query images of 519 persons, and 17,661 gallery images of 1,110 persons. Occluded-ReID (Zhuo et al. 2018) consists of 1,000 occluded query images and 1,000 full-body gallery images both belonging to 200 identities. Partial-ReID (Zheng et al. 2015b) involves 600 images from 60 persons, and each person consists 5 partial and 5 full-body images. Market-1501 (Zheng et al. 2015a) contains 12,936 training images of 751 persons, 19,732 query images and 3,368 gallery images of 750 persons captured by six cameras.

Evaluation Metirc. All methods are evaluated under the Cumulative Matching Characteristic (CMC) and mean Average Precision (mAP). Floating Point Operations (FLOPs) represents the amount of model computation.

Experimental results on Occluded ReID Datasets. In table 1, we compare FPC with state-of-the-art methods on two occluded ReID datasets (i.e., Occluded-Duke and Occluded-ReID). Methods in different categories are compared, including the partial ReID methods (He et al. 2018), key-points based methods (Miao et al. 2019; Wang et al. 2020a, 2022a; Hou et al. 2021), data augmentation methods (Chen et al. 2021; Wang et al. 2020a, 2022b), transformer-based methods (Li et al. 2021; He et al. 2021), gallery-based reconstruction methods (Yu et al. 2021; Xu et al. 2022). In addition, TransReID (He et al. 2021) and PFD (Wang et al. 2022a) use camera information. Since no training set is provided for Occluded-ReID, we adopt Market-1501 as the training set the same as other methods to ensure the fairness of comparison. We can see that our FPC outperforms existing approaches and demonstrates the effectiveness for the occluded ReID tasks. Specifically, our FPC achieves the best performance on the challenging Occluded-Duke dataset, outperforming other methods by at least 6.0% Rank-1 accuracy and 8.6% mAP. On the Occluded-ReID dataset, FPC produces the highest mAP, outperforming the other methods by at least 1.6%. FPC achieves comparable results in Rank-1 accuracy with FED (Wang et al. 2022b), and much better than others.

Experimental Results on Holistic and Partial ReID Datasets. Many existing occluded ReID methods can not be effectively applicable to holistic and partial ReID datasets. On the contrary, FPC achieves great performance improvement on holistic and partial datasets, i.e., Market-1501 and Partial-ReID. The results are shown in table 2. We compare FPC with three categories of methods, including the holistic ReID methods (Sun et al. 2018), the current leading methods in occluded ReID (Miao et al. 2019; Wang et al. 2020a; Chen et al. 2021; Li et al. 2021; He et al. 2021; Wang et al. 2022a, b), the feature reconstruction based methods in occluded ReID (Hou et al. 2021; Yu et al. 2021; Xu et al. 2022). On the Partial-ReID dataset, FPC achieves the best mAP result, lower than the PAT and FRT in Rank-1 accuracy. We consider that we adopt the ViT models and are more prone to overfit on small datasets which leads poor cross-domain generalization. We also observe that FPC achieves competitive Rank-1 accuracy and the highest mAP on the Market-1501 dataset, which is at least 1.7% mAP ahead of other methods. The excellent performance on holistic and partial ReID datasets illustrates the robustness of our FPC.

Analysis of proposed components. The results are shown in table 3. Index-1 is our baseline architecture. To assess the effectiveness of $\mathcal{S}$, we conduct a comparison between index-1 and index-2. Our findings demonstrate that $\mathcal{S}$ leads to a 3.4% improvement in Rank-1 accuracy and a 3.5% increase in mAP over the baseline. This suggests that $\mathcal{S}$ has the potential to mitigate the interference of inattentive features (mainly related to occlusion and background noise). Furthermore, $\mathcal{S}$ can also expedite model inference, as elaborated in table 4. By comparing index-2 and index-3, $\mathcal{C}$ improves performance by 7.2% Rank-1 accuracy and 11.5% mAP, which indicates that $\mathcal{C}$ can effectively compensate the occluded query feature and bring huge performance improvements. By comparing index-3 and index-4, $\mathcal{M}$ can increase performance by 0.9% Rank-1 accuracy and 1.2% mAP, indicating that the proposed distance metric contributes to the find the accurate gallery neighbors.

Analysis of Distance Weight $\alpha$. The distance weight $\alpha$ defined in eq. 10 balances the importance between image-level and patch-level distance. From fig. 3(a), as $\alpha$ goes from 0 to 1, the patch-level distance gradually takes effect. When $\alpha$ is 0.4, the combination of both achieves the best performance with 76.7% Rank-1 accuracy and 72.8% mAP.

Analysis of Number $K$ of Nearest Gallery Neighbors. We conduct quantitative experiments to find the most appropriate number $K$ of nearest gallery neighbors. We conclude from fig. 3(b) that too small a choice of $K$ lacks sufficient information for feature consolidation and too large a choice increases the risk of incorrectly selecting neighbors. When the value of $K$ is 10, we achieve the optimal performance.

Analysis of Keep Rate $\gamma$ in Sparse Encoder. Here, keep rate $\gamma$ reflects the number of preserved tokens in each layer of the sparse encoder. We find instructive observations from table 4: as the $\gamma$ goes from 1.0 to 0.8, the FLOPs drops while the model performance improves. This suggests that a reasonable choice of $\gamma$ can effectively filter out inattentive features, thus reducing the computational complexity and enhancing the model inference capability. When $\gamma$ is 0.8, we achieve the best balance between computational complexity and experimental performance, which leads to an improvement of 0.5% mAP, 0.7% Rank-1 accuracy and 25% reduction in FLOPs.

Analysis of Patch Drop Strategy in Sparse Encoder. We experiment with three variants of patch drop approaches to demonstrate the effectiveness of the proposed method. As shown in fig. 4, we preserve the same number of patches and choose different preservation methods: Random Drop means that $\mathcal{K}$ patches are randomly selected to be retained. Non-salient Drop, as the proposed method, preserves the patches corresponding to the $\mathcal{K}$ largest values in the $\mathrm{[cls]}$ token attention. Conversely, Salient Drop preserves the $\mathcal{K}$ smallest ones. The performance of the three patch drop approaches with different keep rate is shown in fig. 5. We observe that the proposed Non-salient Drop method achieves best performance under different keep rate settings, indicating the importance of attentive features for feature matching and consolidation. In addition, the result of random drop test still achieves relatively high performance, which reflects that our proposed sparse encoder structure is robust against information loss.

Analysis of speed and storage optimization. Our approach relies on gallery information for feature consolidation, to address the practical concerns regarding speed and storage for real-world problems, we conduct analysis on our optimization strategies on the Occluded-Duke dataset. In $\mathcal{S}$, token sparsification reduce 25% Flops (in table 4) with $\gamma$ equals 0.8 and thus accelerates model inference. In $\mathcal{M}$, we use a two-step matching procedure, i.e., use cosine similarity for initial filtering and compute EMD in the finer step, achieving a remarkable 138-fold reduction in computational costs, compared to the exhaustive search based on EMD. Our approach takes 0.43ms per image, while the full EMD calculation takes 60.0ms per image. Furthermore, if we replace full image tokens with $\mathrm{[cls]}$ token, our approach saves a lot memory (i.e., only takes 0.05G for the entire gallery set) and achieves 17x faster in nearest neighbor search time (i.e., from 0.43ms to 0.026ms per image), while still preserving an acceptable degree of precision (i.e., mAP: 72.8% to 70.2%, Rank-1: 76.7% to 74.3%, which still outperforms others).

Comparisons with Re-ranking. An alternative approach is the re-ranking technique (Zhong et al. 2017), which also uses k-nearest gallery neighbors information. We compare $\mathcal{C}$ with re-ranking (Zhong et al. 2017), the second group results in table 5 indicate that re-ranking fails to attain comparable performance of $\mathcal{C}$, lead to a reduction of 3.2% Rank-1 and 0.3% mAP. Further, our $\mathcal{M}+\mathcal{C}$ is approximately 17 times faster than re-ranking, with respective times of 0.47ms and 7.9ms per image. Moreover, the last group shows $\mathcal{C}$ and re-ranking can be jointly employed and outperforms others.

Patch pruning. The patch-drop process in different layers of sparse encode is shown in fig. 6. We observe that with the increasing number of sparse encoder layers, more object occlusion, pedestrian occlusion and background noise are filtered out, while the essential classification information of target pedestrians is preserved.

We show the visualization of patch drop with different keep rate $\gamma$ in fig. 7. $\gamma$ reflects the number of preserved patches in the sparse encoder. As $\gamma$ decreases from 0.9 to 0.5, the patches corresponding to backgrounds and occlusions are gradually pruned. When the $\gamma$ is 0.5, only important target human areas and several outliers are preserved.

We show more visualization results in fig. 8 in order to illustrate the attentive token identification. Input images are chosen from Occluded-Duke(Miao et al. 2019), including object occlusion and pedestrian occlusion. The results validate the ability of our sparse encoder to be applicable to a variety of occlusion cases.

In our paper, we discuss the effect of token sparsification, i.e., , preserving attentive tokens and dropping inattentive tokens, which can both reduce model computation and enhance model inference. Similarly, this sparsification mechanism is also applicable to other occluded ReID methods. Here we demonstrate the effectiveness of token sparsification with Transreid (He et al. 2021) on Occluded-Duke(Miao et al. 2019) and Occluded-ReID(Zhuo et al. 2018) datasets, as shown in table 6. We discover that the token sparsification (abbreviated as T.S.) improves the performance of Rank-1 accuracy and mAP on both datasets, which demonstrates that token sparsification may have broad scope of application in occluded ReID tasks.

We have performed an ablation study on the token sparsification layers, as detailed in  table 7. Our results indicated that the foremost layer has a significant impact on performance, and an excessive forward shift of the layer may lead to performance degradation.

We adopted a two-phase training strategy to optimize computational memory efficiency. Initially, we focused on training the sparse encoder. Upon completion of this phase, the parameters of the sparse encoder were fixed, and we proceeded to train the parameters of the transformer decoder module. The first stage effectively circumvents memory constraints associated with large gallery memory storage. Subsequently, the second stage demonstrated a reduced need for epochs to achieve effective convergence. Furthermore, we input only the tokens of gallery neighbors into the transformer decoder for model inference.

We recognize that there are many challenges to overcome before our approach can be applied to real-world Re-ID scenarios. It is important to note that these challenges are not unique to our approach, but rather common issues faced by most existing person Re-ID methods. These challenge are mainly: 1) large number of gallery images; 2) the gallery needs update; 3) expensive cost of feature storage; 4) no positive samples for a query image.

Classic person Re-ID techniques, such as pair-wise similarity calculation (Ye et al. 2021) or gallery feature reconstruction/aggregation (Xu et al. 2022; Yu et al. 2021; Liao and Shao 2021; Wu, Zhu, and Gong 2022), also face the same issues of large gallery size, gallery update requirement, high cost of feature storage, and absence of positive samples in the gallery for a query image. Moreover, for 1) and 3), in our implementation, the feature size is 243x768 compared to the gallery image size 224x224x3, the extra feature storage is still in the same order of magnitude. For 2), our approach treats each image in the gallery as independent during matching and consolidation, so an update of the gallery would just simply involve deleting and inserting operations. For 4), If there are no positive samples in the gallery for a query image, it is impossible for person Re-ID approaches, including ours, to find the correct identity, as it would violate the pre-conditions for the task. In such cases, it is likely that our approach would use the closest negative samples for consolidation and report a low confidence score.

In addition, it should be noted that our approach still has the potential to be applicable to real-world scenarios. To address the limitations outlined in points 1) and 2), we can employ the efficient HNSW algorithm (Malkov and Yashunin 2018), which has demonstrated fast search speeds of 94ms on a million-scale SIFT dataset and 18.3s on a million-scale ImageNet dataset. To tackle point 3), we can replace fine-grained patch tokens with a $\mathrm{[cls]}$ token, which greatly reduces memory storage requirements while still maintaining an acceptable level of accuracy, as in paper Sec. 4.3. Finally, in regards to point 4), we can adaptively adjust the size of neighbors based on EMD distance rather than relying on a fixed number of neighbors. Nevertheless, our framework is robust to the use of incorrect neighbor features, as demonstrated by our results in Fig. 3(b) of the paper, where an over-sized choice of neighbors (K=14) had only a minimal impact on performance (a decrease of 1.1% in mAP and 2.1% in Rank-1 compared to the best results).

As described in Sec. 3.4 of the paper, our method uses a two-step procedure to efficiently select optimal samples. The first step involves finding 100 candidate samples using cosine distance and the second step integrates EMD to determine the final optimal samples. Our approach showed its efficiency on the Occluded-Duke dataset, where computing EMD for all 17,661 gallery images took 1152.5s, while our two-stage process took only 11.0s, as presented in  table 9. This is in comparison to the k-reciprocal re-ranking process commonly used in Re-ID, which takes 157.1s, making our EMD computation highly efficient.

We acknowledge that the patch drop rate in our model is currently fixed as a potential limitation. Our paper ablation studies on the keep ratio have shown that a range of 0.7-0.9 consistently yields superior results on Occluded-Duke. While not currently implemented, future research plans include the adaptive optimization of the drop rate.

In the future, we envisage investigating the proposed pruning and consolidation framework across a wider spectrum of occlusion challenges, including but not limited to mixed holistic and occluded ReID, occluded object classification, among others. This endeavor is aimed at bolstering the generalizability and applicability of our framework.