Unsupervised Person Re-Identification with Multi-Label Learning Guided Self-Paced Clustering

Most existing person Re-ID methods employ supervised model learning on per-camera-pair manually labeled pairwise training data. The main techniques of these methods are focused on distance metric or subspace learning [4, 46], view-invariant discriminative feature learning [25, 37, 56]. Based on the surge of deep learning techniques, the field of supervised person Re-ID has witnessed rapid progress in recent years [62, 23, 1]. Generally, most of these methods assume that person images are well-aligned. In fact, it is difficult and impractical to given perfect annotation information while person poses are changing. To overcome this limitation, lots of works have adopted attentional deep learning approaches to tackle the misalignment problem. However, supervised learning methods rely on substantial costly and time-consuming labeled training data, which limits those approaches’ scalability and practicability. Although these methods show certain generalization ability on labeled data, supervised Re-ID methods still lacks of effective and practical applications on unlabeled data.

With the progress of deep learning on unsupervised feature learning, researchers begin to apply deep learning to address unsupervised person Re-ID tasks [27, 43]. The open set domain adaptation has been extensively applied to solve the problem of image classification tasks [31, 11], where several classes are unknown in the two domains (or in the target domain). Recently, the domain adaptation strategy has been also widely used for unsupervised person Re-ID tasks [6, 32, 39]. Most of these UDA methods refer to transfer learning to learn the knowledge from the source dataset and transfer them to target dataset. It is clear that the former is an auxiliary and necessarily labelled dataset but the latter is a unlabelled dataset [52, 42, 21, 51]. However, classes of the two domains are entirely different for UDA in person Re-ID, which presents a greater challenge.

To address this domain adaptation problem, there are three typical methods as follows. The first category of methods explore image-style transformation from labeled source domain to unlabeled target domain [6, 39, 41]. In [41], PTGAN enforces the self-similarity of an image before and after translation and the domain dissimilarity of a translated source image and a target image. In paper [62], Zhong et al. first propose a Hetero-Homogeneous Learning (HHL) method to learn camera-invariant network for the target domain. However, HHL overlooks the latent positive pairs in the target domain, which might lead the Re-ID model to be sensitive to the background or pose variations in the target domain. To overcome these drawbacks, Zhang et al. [55] propose a self-training method with a progressive augmentation framework to promote the model performance progressively on the target dataset.

The second category of methods utilize a model distillation by using the teacher model to guide learning of the student model. A vast majority of knowledge distillation methods are adopted for Re-ID problems to alleviate cross-camera scene variation explicitly or implicitly [13, 42]. In [42], a multi-teacher adaptive similarity distillation framework is proposed to learn a user-specified lightweight student model from multiple teacher models, without access to source domain data. Wu et al. [43] propose to learn consistent pairwise similarity distributions for intra-camera and cross-camera matching with the guidance of prior common knowledge of intra-camera matching.

The third category of methods attempts on optimizing the Re-ID model with soft labels for target-domain samples by measuring the similarities with reference images or features. Zhong et al. [63] investigate the impact of intra-domain variations and impose three types of invariance constraints on target samples. They assign soft labels and minimize the target invariance by proposing an exemplar memory module [16, 40], which caches feature vectors of every instance. And then, Yu et al. [51] propose a method called deep soft multilabel reference learning (MAR) method, which conducts multiple soft-label learning by comparing with a set of reference persons. Ge et al. [14] designed an asymmetrical framework to generate more robust soft labels via the mutual mean-teaching. However, these methods need to learn one or several effective teacher model(s), which only depend(s) on the diversity and quality of source domain data. It can’t be ignored that they still need a labeled source data, and they don’t explore the sample similarity between the source and the target domain. However, our method not only discard the requirment of labeled source domain data, but also could mine sample similarities on target domain data directly. Specially, we leverage MN to mine multi-granularity person features and ML module to store these up-to-date multi-scale features for the whole target data.

Unsupervised learning methods have attracted much attention because of their capability of saving the cost of manual annotations. The main task of unsupervised learning has shifted to how to fully mine the useful information in the increasingly growing unlabeled dataset. Unsupervised learning methods for person Re-ID generally involve two aspects: traditional unsupervised methods and clustering-guided deep learning methods.

Traditional unsupervised person Re-ID mainly focus on feature learning, which created hand-craft features [2] that can be utilized directly to the unlabeled dataset. In the earlier works, Liao et al. [25] propose the local maximal occurrence (LOMO) descriptor, which includes the color and SILTP histograms. Zheng et al. [58] propose to extract the 11-dim color names descriptor for each local patch, and aggregate them into a global vector through a Bag-of-Words model. It is worth pointing out that these methods’ performance are not effective and satisfactory on unlabelled dataset. The latest studies suggest that it is difficult to learn robust and discriminative features by traditional unsupervised methods.

With the surge of deep learning techniques, recent studies have focused on the clustering-guided deep learning methods for unsupervised Person Re-ID [52, 50, 38]. Because of the most important is that clustering method is intuitive and efficient for unsupervised machine learning. These methods generally focused on generates pseudo-labels on the target domain and then use these pseudo-labels to learn deep models in a supervised manner. Liu et al. [29] propose a stepwise metric promotion approach to refine the pseudo labels by iteratively estimate the annotations of training tracklets. Wu et al. [45] propose a progressive sampling method to gradually predict reliable pseudo labels and uncover the unlabeled data for one-shot video-based Re-ID. Yang et al. [47] introduced the asymmetric co-teaching strategy to refine the pseudo-labels in the clustering-based method. Zhai et al. [54] present a novel augmented discriminative clustering technique that incorporates style-translated images to improve the discriminativeness of instance features. However, such methods rely on a good deep Re-ID model as an initialized feature extractor for unsupervised learning. Aside from these methods that require an auxiliary Re-ID model, they also face the hard or difficult samples to effect the quality of the label predict. Therefore, some researchers first consider focusing on the fully unsupervised Re-ID task without rely on any initialized model.

Different from previous works, Lin et al. [27] proposed a bottom-up clustering framework that iteratively trains a network based on the pseudo labels generated by unsupervised clustering. It not only considers the diversity over each sample but also exploits the similarity within each class. Ding et al. [7] designs a novel dispersion-based clustering approach which can discover the underlying feature space for unlabeled pedestrian image data. Zeng et al. [53] propose a hierarchical clustering with hard-batch triplet loss. These approaches have explored cluster distributions in the target domain. They still face the challenge on how to precisely predict the label of hard samples. On Different from these methods which classify each image into a single class, the multi-label classification has the potential to exhibit better efficiency and accuracy. For example, Lin et al. [28] proposed an unsupervised Re-ID network that softened labels to reflect the image similarity and eliminated the hard quantization error. Wang et al. [34] consider the unsupervised person ReID as a multi-label classification task to iteratively predict multi-class labels and update the network with multi-label classification loss. However, this strategy of multi-label keeps all the samples in the memory which may introduce noisy samples in training phase. Different from all the above methods, our method addresses the fully unsupervised person Re-ID problem with joint multi-label and self-paced clustering.

To tackle the unsupervised person Re-ID, we propose a multi-label learning guided self-paced clustering (MLC) framework as shown in Figure 2. Our MLC framework includes three crucial modules, namely a multi-scale network (MN), a multi-label learning (ML) module, and a self-paced clustering (SC) module. Given an image, the multi-scale network extracts both global features in the whole image and local features in sub regions by global max pooling. The ML module leverages an updatable memory bank to assign multi-hot labels for all unlabeled images and uses multi-label classification loss for training. Commonly, the memory bank is composed of the multi-scale features of all samples in a dataset. The multi-hot label for an image is determined by the similarities between the image feature and memory bank. The SC module filters noisy samples and assign pseudo labels by the step-wise clustering and sampling. In practice, we first perform several epochs of multi-label training to ensure every image is used for training and then we jointly apply SC and ML which may trade-off the noisy samples and hard samples.

Preliminary. In fully unsupervised person Re-ID tasks, we only have an unlabeled training dataset $X=\left\{x_{1},x_{2},\cdots,x_{N}\right\}$ containing $N$ person images. Our purpose is to learn a discriminative feature extractor $\phi(\theta;x_{i})$ from $X$ without any available annotations. The parameters of $\phi$ are optimized iteratively using an objective function. This feature extractor can be applied to the gallery set, $G=\left\{g_{1},g_{2},\cdots,g_{N_{t}}\right\}$ of $N_{t}$ images, and the query set $Q=\left\{q_{1},q_{2},\cdots,q_{N_{q}}\right\}$ of $N_{q}$ images. During the evaluation, for any query person image $q$, the feature extractor is expected to produce a feature vector to retrieve image $g$ containing the same person from a gallery set $G$. In other words, we need to use the features of a query image $\phi(\theta;q_{i})$ to search more similar feature with $g$ from the gallery set $G$. Hence, it is critical to learn a disciminative feature extractor $\phi(\theta;\cdot)$ for person Re-ID model. The conceptual optimization goal of distance between each pair of images is defined as,

$$\hat{g}=arg\min_{g\in G}dist(\phi(\theta;q_{i}),\phi(\theta;g_{i}))$$

(1)

where $dist(\cdot)$ is the distance metric, e.g., the L2 distance. Generally, person Re-ID mainly adopts Euclidean or cosine distances as the re-ranking method for the retrieval stage. In this case, we use the Euclidean as the distance metric function to re-rank the ReID results. More important, we will bring in a k-reciprocal encoding method with the Jaccard distance of probe and gallery images to computer the distance metric as the re-ranking method [61] for our self-paced clustering. The detailed application will be introduced in Section 3.5.

Initialization with hard labels. In order to learn a discriminative feature extractor, the transitional supervised learning method needs person identity labels for each image. However, there are no manually annotated labels in fully unsupervised Re-ID tasks, so we need to generate the pseudo labels instead of that. Thus, we start by treating each training image $x_{i}$ as an individual class, and initially assign $x_{i}$ with a label $y_{i}$ by its index, i.e, $Y=\left\{y_{1},y_{2},\cdots,y_{N}\right\}$. The feature extractor $\phi(\theta;\cdot)$ is appended by a classifier $f(w;\phi)\in\mathbb{R}^{N}$ parameterized by $w$. The optimization is defined by the following objective function:

$$\min_{\theta,w}\sum_{i=1}^{N}\l(f(w;\phi(\theta;x_{i})),y_{i}),$$

(2)

where $l$ is the cross-entropy (CE) loss for classification. The hard labels Y are suitable for initialization but not reliable for unsupervised feature learning. We warm up the neural network with the initialized hard label, allowing the model to reach a certain local optimal field where subsequential approaches need to be explored.

As the feature extractor, the multi-scale network aims to capture multi-granularity person features for similarity computing. Specifically, we use the base of ResNet50 as our backbone since it is widely adopted in person Re-ID tasks and obtains better performance. Inspired by the recent domain adapted Re-ID work [12], we compute the similarity between two persons not only by global information from the whole body but also by local information from different parts of a person. The detailed architecture of our MN is illustrated in Figure 3.

We remove the down-sampling operations before $res\_conv5\_1$, and uniformly split the feature maps into an upper and an bottom part at the last $conv$ layer. We utilize global average pooling (GAP) operation on the whole final feature map and the partial feature maps to obtain three feature vectors for each image, i.e., $f_{g},f_{up},\textrm{and}~{}f_{low}$. To obtain better multi-granularity features, we concatenate them as a final person representation $f_{all}$.

The multi-label learning (ML) module aims to learn discriminative features by assigning similar multi-hot labels for similar images based on a feature memory bank.

Memory Bank. A memory bank consists of the representations of all samples in the dataset. Following [34, 16, 40], we maintain the memory bank to serve as a feature storage that saves up-to-date features of the training dataset. Memory bank allows the network discovering more negative samples from the memory buffer to pair with positive sample without recomputing their features Compared with previous methods, the benefit of a memory bank is to collect more feature informative pairs with the cost of memory space for features stored in the memory bank.

Initialization. We initialize the memory module $\boldsymbol{M}$ by computing the features of a set of randomly sampled training images based on the warm-up model. Formally, $\boldsymbol{M}=\left\{(f^{1}_{all}),(f^{2}_{all}),\cdots,(f^{N}_{all})\right\}$, where the $\boldsymbol{M}\in\mathbb{R}^{N\times d}$ and $d$ is the dimension of these features. The $f^{i}_{all}$ is initialized as the feature of the $i$-$th$ sample $x_{i}$. In the memory bank, each slot $\boldsymbol{M\left[i\right]}$ stores the L2-normalized feature $f^{i}_{all}$ in the key part, while storing the hard label in the value part.

Updating. During each training iteration, the feature vectors on each mini-batch would be involved in memory bank updating. The whole unlabeled data can be cached in the memory bank, which is dynamically updated with features computed in the training. We update the memory bank in a running-average manner as follows,

$$\begin{array}[]{l}\boldsymbol{M\left[i\right]^{t}}\leftarrow\alpha\boldsymbol{M\left[i\right]^{t}}+(1-\alpha)f^{i}_{all},\\ \boldsymbol{M\left[i\right]^{t}}=\boldsymbol{M\left[i\right]^{t}}/||\boldsymbol{M\left[i\right]^{t}}||_{2},\end{array}$$

(3)

where the superscript $t$ is denoted as the current iteration epoch, $\alpha$ is the momentum of updating rate. Then, we utilize the Memory-based Positive Label Prediction (MPLP) method [34] with the hard label to predict the multi-hot label $\bar{y}$ based on the memory bank $\boldsymbol{M}$. Followed the MPLP, we first computes a rank list $R_{i}$ to store the similarities between a sample and the memory bank as follows,

$$R_{i}=\mathop{\textup{arg}\,\textup{sort}}_{j}(s_{i,j}),j\in[1,N],$$

(4)

$$s_{i,j}=\boldsymbol{M\left[i\right]^{T}}\times\boldsymbol{M\left[j\right]}$$

(5)

where $s_{i,j}$ is the similarity score of $x_{i}$ and $x_{j}$, and the $R_{i}$ aims to find the candidates for reliable labels about $x_{i}$. We use a similarity threshold and a cycle consistency scheme to select relevant label candidates and filter hard negative labels. The positive label is set as,

$$P_{i}=\hat{R}_{i}[1:l]$$

(6)

where $\hat{R}_{i}$ is the $top$-$k_{i}$ nearest labels and $\hat{R}_{i}={R}_{i}[1:k_{i}]$, ${R}_{i[k_{i}]}$ is the last label with similarity score higher than a threshold which decide the quantity of label candidates. The $l$ satisfies $i\in R_{\hat{R}_{l}}\&i\notin R_{\hat{R}_{l+1}}$. As $P_{i}$ contains $l$ labels, $x_{i}$ would be assigned with a multi-label $\bar{y_{i}}$ in which value 1 indicate positive classes,

$$\bar{y_{i}}=\left\{\begin{array}[]{l}\ \ 1\quad j\in P_{i}\\ -1\quad j\notin P_{i}\end{array}\right.$$

(7)

The multi-label classification loss, which is computed on positive classes and sampled hard negative classes, is shown below.

$$\mathcal{L}_{mmcl}=\sum_{i=1}^{N}\frac{\delta}{\left|P_{i}\right|}\sum_{p\in P_{i}}\mathit{l}(p|x_{i}))+\frac{1}{\left|S_{i}\right|}\sum_{s\in S_{i}}\mathit{l}(s|x_{i}))$$

(8)

where $S_{i}$ is the collection of hard negative classes for $x_{i}$, we also select the $top$-$r\%$ classes as the hard negative classes, and $\left|S_{i}\right|=(N-\left|P_{i}\right|)\cdot r\%$. The $\delta$ is a a coefficient measuring the importance of multi-label classification loss.

As mentioned above, the multi-label learning strategy keeps all samples for training which may hurt feature learning due to noisy samples. The self-paced clustering (SC) module is proposed to update the training dataset with more clean samples and assign pseudo labels for training.

As shown in the bottom-right of Figure 2, we firstly employ the SC module to cluster the final similarity features extracted by our MN. The clustering method aims to group similar entities together after computing the distance metric with k-reciprocal encoding [61] for all training samples. The sampling is used to filter noisy samples and assigns pseudo-labels $\tilde{y}_{i}$ according to the group entities. Finally, we form a new training dataset with pseudo-labels to leverage SC and ML module for jointly learning.

For clustering methods, the selection of nearest neighbors is crucial. It is used for merging all the instances into right cluster and finally affects the clustering results and the quality of pseudo labels. The conventional k-means clustering algorithm comes as a natural choice which selects nearest neighbors according to the distance of a sample to the centroids of clusters. As an alternative choice, the DBSCAN [8] algorithm can be used which selects nearest neighbors by Jaccard distance [61] matrix and k-reciprocal nearest neighbor method. We use DBSCAN as our default SC module, and apply step-wise clustering and sampling to remove the noisy samples of the whole dataset. During the progressive clustering stage, we try to keep the most reliable clusters. In order to avoid accumulating training error caused by noisy clusters, we remove these noisy samples and constitute a new training set with pseudo labels for joint training.

Since labels are available after self-paced clustering, we treat the training process as a classification problem. Specifically, we adopt Cross-Entropy (CE) loss for classification and triplet loss for metric learning. We apply CE loss with label smoothing as follows,

$$\mathcal{L}_{CE_{p}}=\frac{1}{N}\sum_{i=1}^{N}\l(f(w;\phi(\theta;x_{i})),\tilde{y}_{i}^{s})$$

(9)

where the smooth j-th item $\tilde{y}_{j,i}^{s}=1-\varepsilon+\frac{\varepsilon}{C}$ if $j=\tilde{y}_{i}$, otherwise $\tilde{y}_{j,i}^{s}=\frac{\varepsilon}{C}$. $C$ is the numbers of predict the identities by SC module. $\varepsilon$ is a small constant for the label smoothing.

Formally, the triplet loss function is defined as follows,

$$\begin{split}\mathcal{L}_{tri}=\frac{1}{N}\sum_{i=1}^{N}\max(0,\left\|f(w;\phi(\theta;x_{i}))-f(w;\phi(\theta;x_{i,p}))\right\|\\ +m-\left\|f(w;\phi(\theta;x_{i}))-f(w;\phi(\theta;x_{i,n}))\right\|)\end{split}$$

(10)

where $\left\|\cdot\right\|$ denotes the $L_{2}$-norm distance, the $x_{i,p}$ and $x_{i,n}$ indicate the hardest positive and hardest negative feature sample in each mini-batch, and $m$ denotes the triplet distance margin, and sets the default value to 0. The overall loss function for optimization is the combination of multi-label classification loss and the pseudo label classification loss (CE loss and triplet loss) as follows,

$$\mathcal{L}_{o}=\lambda_{1}\mathcal{L}_{mmcl}+\lambda_{2}(\mathcal{L}_{CE_{p}}+\mathcal{L}_{tri}).$$

(11)

We evaluate our approach on three large-scale bench-mark datasets: Market1501 [58], DukeMTMC-reID [60] and MSMT17[41].

Market1501 Datasets Market1501 contains 1,501 person identities with 32,668 images which are captured by six cameras. It contains 12,936 images of 751 identities for training and 19,732 images of 750 identities for testing.

DukeMTMC-reID Datasets DukeMTMC-reID is a subset of the DukeMTMC dataset. It has 1,404 person identities from eight cameras, with 36411 labeled images. It contains 16,522 images of 702 identities for training, and the remaining images out of another 702 identities for testing, including 2,228 images for query, and 17,661 images for gallery.

MSMT17 Datasets MSMT17 is the most challenging and the currently largest-scale dataset which contains 126,441 images of 4,101 person identities captured from 15 camera views. It is spitted to 32,621 images of 1,041 identities for training, and 93,820 images of 3,060 identities for testing.

Evaluation Metrics We followed the standard training/test split and evaluated the single-query test evaluation settings. For the evaluation metrics, we used the Rank-1/Rank-5 matching accuracy, which means the query picture has the match in the top-k list. And we use the mean Average Precision (mAP), which is computed from the Cumulated Matching Characteristics (CMC) [15].

Implementation Details We implement our method with Pytorch. For data pre-processing, input images are resized into $256\times 128$. We apply some commonly-used data augmentation methods including random horizontal flipping, random cropping, color jitter, random erasing, and CamStyle [64]. For a fair comparison, we adopt ResNet-50 pre-trained on ImageNet as our backbone network. The batch size is set to 128 for training. We use the standard SGD as an optimizer, the momentum is 0.9, and weight decay is $5\times 10^{-4}$. The initial learning rate is 0.1. We train the model for $60$ epochs, and the learning rate divided by 10 after every $30$ epochs. Following [34], we initialize the memory bank as all zeros, and use the hard label to warm-up model and the memory is fully updated for $5$ epochs. The memory updating rate $\alpha$ starts from 0 and grows linearly to 0.5. The similarity threshold in ML module is 0.6. We jointly train the ML and SC modules after $15th$ epoch.

To investigate the effect of the joint learning based on our proposed MLC, we compare MLC with three baselines: 1) the first is basic supervised learning method [25] with triplet loss on the labeled data; 2) the second is direct feature evaluation with the pre-trained ResNet-50 model on ImageNet; 3) the third is unsupervised learning with multi-label classification loss [34]. The experimental results are shown in Table 1. The first two baselines represent the upper and lower bound performance of the backbone model. The third baseline utilizes similar multi-label prediction with ours for discriminative feature learning. In theory, when our generated pseudo labels are closer to the truth labels, the performance of our MLC will approach to the fully-supervised baseline1. For fair comparisons, we also present the results obtained without MN module, i.e., the same feature extractor with baseline methods.

As can be seen, the baseline1 achieves the highest performance with supervised learning, $e.g.$, 70.7% mAP on Market-1501 and 58.0% mAP on DukeMTMC-reID. On the contrary, the performance of baseline2 is very poor which shows that there is a large gap between person Re-ID and the object classification of ImageNet. Based on the predicted multi-label, the baseline3 boosts the baseline2 significantly, and achieves 42.5% and 38% improvements in mAP on Market-1501 and DukeMTMC-reID, respectively. Without the multi-scale network, our MLC already outperforms all baseline methods largely, with 62.6% mAP on Market-1501 and 50.0% mAP on DukeMTMC-reID. By considering the global and local information (i.e., with MN), the MN module further boosts performance by 3.6% mAP on Market-1501 and 2.3% mAP on DukeMTMC-reID.

We compare our approach with the state-of-the-art unsupervised learning methods for person Re-ID including: 1) the hand-crafted features (including LOMO [25] and BoW [58]; 2) the pseudo label learning methods without any other labeled dataset (e.g. BUC [27], SSL [28], DBC [7], HCT [53], MMCL [34]); 3) the unsupervised transfer learning and domain adaptation approaches (e.g. UMDL [32], CAMEL [50], PUL [9], PTGAN [41], SPGAN+LMP [6], MMFA [26], TJ-AIDL [39], HHL [62], ECN [63], MAR [51], PAUL [49], SSG [12], CR-GAN [3], CASCL [43], PDA-Net [24], MMCL [34], ADTC [18], and D-MMD [30]). The comparison results on Market-1501 and DukeMTMC-reID are presented in Table 2, and the comparison on MSMT17 is listed in Table 3.

From Table 2, we observe that our MLC method consistently outperforms recent state-of-the-art methods on both Market-1501 and DukeMTMC-reID with or without source dataset. Without source dataset, the hand-crafted feature based methods show the worst performance since the representation ability of designed features is limited. These deep learning methods (from BUC to MMCL) with pseudo labels significantly outperform the hand-crafted feature based methods which indicates that pseudo labels and deep networks are effective. Our MLC achieves the best results with mAP 66.2% and 52.3% on Market-1501 and DukeMTMC-reID. Compared with pure clustering based methods like SSL, our MLC leverages the ML module to learn better initial representations to guide subsequent clustering. Compared with the MMCL method, our MLC uses another SC module to generate new dataset and refine pseudo labels which largely enhances the Re-ID models through joint training.

We further compare our method with unsupervised transfer learning and domain adaptation methods. Our MLC not only surpasses those fully unsupervised methods but is also better than these unsupervised transfer learning and domain adaptation methods. Specifically, under the transfer learning setting, our method achieves the best performance on both Market1501 and DukeMTMC-reID. For example, our MLC obtains mAP 65.9% and 55.0% on Market1501 and DukeMTMC-reID, respectively. An interesting observation is that the results of our MLC are similar whether there exists annotated source dataset or not. This indicates that our method is suitable for unsupervised person Re-ID learning.

We also conduct experiments on MSMT17, and the results are shown in Table 3. Compared with the other two datasets, MSMT17 is a larger and more challenging dataset because of more complex lighting and scene variations. A limited number of researches have reported performance on MSMT17, including unsupervised learning and transfer learning with domain adaptation methods, such as MMCL [34], PTGAN [41], ECN [63], and SSG[12]. As shown in Table 3, our approach outperforms existing methods by large margins under both unsupervised and transfer learning methods. We can see that our MLC without MN module obtain the best performance on unsupervised learning method. It achieves mAP 12.0% and outperforms our MLC and MMCL by 0.7% and 1.5% improvements, respectively. Under transfer learning setting, our MLC obtains mAP 15.0% and 16.7% with the source datasets of Market1501 and DukeMTMC-reID. Interestingly, the MLC without MN module achieves better results with mAP 16.2% and 18.0%, respectively. The slightly improve of MN module on MSMT17 may be explained by that the person images of MSMT17 are more variant than the other two datasets in posture and scene. Overall, our MLC improves the state-of-the-arts (i.e. MMCL) by 1.8% and 2.8% in mAP and Rank-1 when using Duke as source dataset.

Evaluation of individual modules in MLC. In order to verify the effectiveness of each module in MLC, we conduct the experiments by evaluating the performance contribution of different modules on the Market-1501 and DukeMTMC-reID dataset. It is summarized in Table 4. We utilize the k-means instead of the DBSCAN as the clustering method in our SC module for comparison. And the number of pseudo identities of k-means clustering are set as 500, 750, 1000. Especially, the cross (or tick) in MN, ML, SC column means the result of MLC without (or with) the corresponding modules, respectively.

Seen in Table 4, when using the k-means instead of the DBSCAN method, the best mAP is 49.3% on Market-1501 and 42.1% on DukeMTMC-reID by $k=750$ and $k=1000$, respectively. Seen from the results, the SC module could improve MLC’s performance obviously on Market-1501 while slightly on DukeMTMC-reID. This shows that even an elementary clustering algorithm could push the SC module, which improves the network to learn more discriminative image features. It can be evidenced that using the SC module could be beneficial to all modules in our MLC. Compared with MLC without the SC module, our MLC is found to significantly boost 22.1% and 10.8% mAP on Market-1501 and DukeMTMC-reID. Especially, it is shown that MLC can achieve the best performance with all modules on both datasets. This result shows, on the one hand, the SC module can remove the noisy samples when joint learning with the ML module. On the other hand, it also can help our MLC to generate more accurate pseudo labels for unsupervised learning.

Evaluation of the parameter $K_{1}$, $K_{2}$ and $K_{sample}$. In the clustering phase, the $K_{1}$ and $K_{2}$ is denoted the parameter in evaluating of the Jaccard distance [61], the former is considered as the contextual knowledge to re-calculate the distance between the probe and gallery, the latter is the number of candidates in the top-k samples of the ranking list about the probe. And the parameter $K_{sample}$ is the minimum sample number of groups in the DBSCAN method. To investigate the properties of our MLC with these parameters, we fix the hyperparameter $\lambda$ as 0.3. And the default setting of $K_{1}$, $K_{2}$ and $K_{sample}$ are 20, 6 and 4, respectively. We evaluate it by increasing $K_{1}$ from 15 to 35, and present the results in Figure 4 (a) and (d). It can be seen that the performance first increases with the growth of $K_{1}$, and then begins a slow decline after $K_{1}$ set to 20. Larger K1 means it is more likely to include false matches in the k-reciprocal set, resulting in a decline in performance. The impact of the size of $K_{2}$ is shown in Figure 4 (b) and (e), it is varied from 2 to 10. We can see that when we set $K_{2}=6$, we get the best performance. Notice that, assigning a too large value to $K_{2}$ also reduces the performance. At last, we evaluate $K_{sample}$ from 2 to 10. Our results are reported in Figure 4 (c) and (f), the performance grows as $K_{sample}$ increases in a reasonable range. Value of $K_{sample}$ greater than 4 will reduce the performance. If the parameter of $K_{sample}$ is set too large, it will affect the performance of DBSCAN clustering.

Evaluation of the hyperparameter $\lambda$. We evaluate how $\lambda$ (the relative importance of multi-label classification loss) affects our model learning. To investigate the properties of our MLC framework, we evaluate different $\lambda$ from 0.1 to 10. The results of rank-1 accuracy and mAP on Market-1501 and DukeMTMC-reID dataset are shown in Figure 5 (a) and (b). We observe that increasing $\lambda$ boosts performance in the beginning, after $\lambda=0.4$ on Market-1501 and $\lambda=0.3$ on DukeMTMC-reID gradually degrades. Too larger $\lambda$ leads to fast degradation, which indicates the pseudo label classification loss is important for joint training our model. As the mater of fact, the result of this experiment is evidenced that the SC joints in training with the ML is a simple and effective strategy for unsupervised person Re-ID tasks.

Qualitative analysis of clustering visualization To better investigate the effectiveness of our MLC, we visualize the clustering results in Figure 6. We illustrate the clustering performance of our MLC with and without the MN module in the right and left clomun, respectively. We also shows the results of MLC (w or w/o the MN module) on Market-1501 and DukeMTMC-reID dataset. The cluster image shows that the MN module could help not only purifying negative samples (images with red bounding box), but also supplementing hard positive samples (images with green bounding box) in each group.This indicates that our MN module can improve the clustering quality by mining the multi-scale features for better similarity measurement.