Chairs Can be Stood on: Overcoming Object Bias in Human-Object Interaction Detection
Supplementary Material

Following previous work [HORCNN_Chao2018WACV, li2020hoi_IDN, zhang2021spatially_SCG_ICCV21], we also report results on the HICO-DET dataset [HORCNN_Chao2018WACV] with Known Object (KO) setting in Tab. LABEL:table:hicodet_ko_pre and Tab. LABEL:table:hicodet_ko, respectively. It can be observed that our method surpasses the baselines under this setting.

The detailed analysis of the coordination of these two classifiers with respect to $\lambda$ is shown in Tab. LABEL:table:lambda. It can be seen both classifiers are essential for the performance improvement.

We compare the memory and computational cost with SCG [zhang2021spatially_SCG_ICCV21] in Tab. 1. Note that the adopted detector (i.e., Faster R-CNN [ren2015fasterrcnn]) is not counted as the detection results can be obtained via one-pass inference for all images before training. It can be observed the overhead brought by our method is negligible for both training and test.

We conducted all experiments on 4 Nvidia 2080Ti GPUs. Due to resource limitation, we reduced the batch size of SCG and QPIC to 8 and linearly scaled their learning rate.¹¹1This may slightly influence the performance and result in inconsistency between reported and reproduced ones. For QPIC, we started from a trained model and finetuned it with the proposed method for a total of 15 epochs. The learning rate is decayed by 0.1 at the 10-th epoch. For the other two baselines, we followed the default scheduling and started the training of $f_{m}$ from the $3rd$ epoch for stability. $\lambda$ is empirically set to 0.4 in all experiments.

For the proposed method, we parameterize $f_{m}$ as a three layer Multi-layer Perceptron with ReLU activation function. For each memory cell, we set the size $n$ to 16 for each object and $k$ to 4. About the write operation, $\tau^{o}$ is set to the third smallest $w^{o}$ (for objects with more than 5 associated verbs) or 0 (other objects). Regarding to other aspects with respect to base models (feature extractor, sampling strategy, and loss function), we adopted their default settings. More details are as follows.

The implementation details of the baselines are listed in Tab. LABEL:table:imp_detail. In this table, the batch size is represented as number of images per GPU $\times$ number of GPUs. BCE stands for binary cross-entropy loss. Kindly find the codes with the corresponding model names in the zip file.

Due to resource limitation, we used a smaller batch size and scaled learning rate for both the baseline (SCG) and our method in all previous experiments. We also studied the performance of baseline and our method under other hyper-parameter settings in Tab. 2. It can be observed that a) smaller batch size results in worse performance (2&4, 3&5). b) linearly scaling learning rate with respect to batch size can prohibit performance degradation to some degree (1&4). c) Under different training settings, our method outperforms the baseline by a considerable margin (2&3, 4&5).

We show the evolution of label distribution under another four randomly picked objects in Fig. LABEL:fig:memvis. It can be observed that the model prefers to sample some frequent class instances at early iterations due to their dominance. When it comes to later training steps, rare class instances gain more attention with the help of the proposed ODM. By the end of the first epoch (i.e.,  4.5k iterations), the tail classes under each object is more frequently sampled.

We provide additional qualitative results in Fig. LABEL:fig:quali_sup. It can be seen that our method can effectively alleviate the object bias problem by reducing false negative errors.