Are Neural Ranking Models Robust?

To evaluate the performance variance of different ranking models, we conduct experiments on several representative ad-hoc retrieval datasets, i.e.,

• •

  MQ2007. Million Query Track 2007 (MQ2007) is a LETOR (Qin and Liu, 2013) benchmark dataset based on the GOV2 collection which includes 25 million documents in 426 gigabytes. Queries for MQ2007 are divided into 5 folds as described in LETOR4.0.

• •

  Robust04. Robust04 contains 250 queries and 528k news articles, whose queries are collected from TREC 2004 Robust Track⁶⁶6https://trec.nist.gov/data/robust.html. There are about 70 relevant documents (news articles) for each query.

• •

  MS MARCO. MicroSoft MAchine Reading COmprehension Document Ranking Dataset (MS MARCO)⁷⁷7https://github.com/microsoft/MSMARCO-Document-Ranking is a large-scale benchmark dataset for web document retrieval. MS MARCO contains 4 million documents and 0.37 million training queries, where each query has one relevant document. Evaluation was on the dev set, which contains 5193 queries⁸⁸8The relevance assessments for the official test set were not public at the time the paper was written..

The detailed statistics of datasets are shown in Table 1. We choose these three datasets according to three criteria: 1) The datasets are public and the original document contents are available, 2) The query numbers are diverse (e.g., 250 queries in Robust04 and 0.37 million queries in MS MARCO), and 3) The literary style of the documents ranges from newswire articles to Web document.

In preprocessing, all the words in the documents and queries are white-space tokenized, lower-cased, and stemmed using the Krovetz stemmer (Krovetz, 2000). For the MQ2007 dataset, we use the data partition and top 40 ranked documents released by the official LETOR4.0 (Qin and Liu, 2013). For the Robust04 dataset, an initial retrieval is performed using the Anserini toolkit with the QL model to obtain the top 100 ranked documents. For the MS MARCO dataset, we use the official top 100 ranked documents retrieved by the QL model.

For traditional probabilistic ranking models, we adopt the static parameters suggested by (Huston and Croft, 2014) on the Robust04 dataset and tune the parameters on the corresponding validation set on the MQ2007 and MS MARCO dataset⁹⁹9$\mu$ in 1-2000 (by 10) for QL and $k_{1}$ in 0.1-5 (by 0.1), $b$ in 0.1-1 (by 0.1) for BM25. . For LTR models, we use standard features released by LETOR 4.0 for MQ2007. In LETOR, there are 46 hand-crafted features in total, among which 42 features are constructed based on the textual elements (e.g., term frequencies, BM25 and language model scores based on title, body, anchor texts, and URL), and 4 features are based on link analysis (e.g., PageRank, inlink number, outlink number, and number of child page). Due to the lack of officially released LTR features, we construct the LTR features for Robust04 and MS-MARCO respectively, by computing the TF, IDF, TF-IDF, Document Length, BM25 score, QL-DIR score and QL-JM scores on the title, url, body and document respectively as the final 28-dimensional LTR features following (Qin and Liu, 2013). For RankSVM, we use the linear kernel with the hyper-parameter C selected from the range [0.0001, 10] on the validation set for three datasets following (Dai et al., 2018). For LambdaMART, we select the number of trees (in 100-500, by 10), leafs (in 2-20, by 2) and shrinkage coefficient (in {0.001, 0.005, 0.01, 0.05, 0.1}) on the validation set for three datasets.

For neural ranking models, the model-specific hyper-parameters are as follows. For DSSM, we use a three-layer DNN and set the node number of each layer as 100, 100 and 50 to avoid overfitting on Robust04 and MQ2007 datasets following (Pang et al., 2017). We set the node number of each layer as 300, 300, 128 on the MS MARCO dataset. For DRMM, we use a four-layer architecture and set the node number of each layer as 30 (histogram bins), 5, 1 and 1 on Robust04 and MQ2007 datasets following (Guo et al., 2016; Pang et al., 2017). We set the node number of each layer as 30 (histogram bins), 10, 1 and 1 on MS MARCO dataset. For Conv-KNRM, we set the n-gram size as 3 and the number of CNN filters as 128. For kernel-pooling, we set the number of kernels to 11 with the mean values $\mu$ of the Gaussian kernels varying from $-1$ to $+1$, and standard deviation $\sigma$ of 0.1 for all kernels (the first kernel’s $\sigma$ is set as 0.001 for exact matching ) on three datasets following (Dai et al., 2018; Hofstätter et al., 2020). For Duet, we used 10 filters with both the local and distributed model, with hidden dimensions set to 30 and 699, respectively on Robust04 dataset following (Yates et al., 2020). We set the filter size as 10 in both local model and distributed model, and the hidden size as 20 in the fully-connected layer on MQ2007 dataset following (Fan et al., 2018). We set the filter size as 32 in both local and distributed model and the hidden size as 32 in the fully-connected layer on MS MARCO dataset. The learning rates of above models are tuned on the corresponding validation set in the range of [1e-5, 5e-3]. For BERT, we choose the fine-tuning learning rate from {5e-5, 3e-5, 2e-5} as recommended in (Kenton and Toutanova, 2019) on three datasets. For ColBERT, we set the fine-tuning learning rate as 3e-6 on three datasets following (Khattab and Zaharia, 2020). For all neural ranking models, we use the Adam (Kingma and Ba, 2014) optimizer. We use LTR and neural ranking models to re-rank the top candidate documents.

To better understand the performance variance, we first analyze the average ranking effectiveness over the queries. Following (Guo et al., 2016; Fan et al., 2018), we take the topic “title” as queries and conduct 5-fold cross-validation on Robust04 and MQ2007 datasets to minimize over-fitting without reducing the number of learning instances. For Robust04 dataset, we use the mean average precision (MAP) (Schütze et al., 2008), normalized discounted cumulative gain at rank 20 (NDCG@20) (Burges et al., 2005), and precision at position 20 (P@20) (Croft et al., 2010) as the evaluation metrics following (Guo et al., 2016). For MQ2007 dataset, we report the P@10 and NDCG@10 following (Fan et al., 2018). Moreover, we also use the recall at position 20 (R@20) (Olson and Delen, 2008) and recall at position 10 (R@10) as the recall oriented evaluation metrics for Robust04 and MQ2007, respectively. For MS MARCO dataset, we report the Mean Reciprocal Rank at 100 (MRR@100 (Radev et al., 2002)), which is suggested in the official instructions. We also report the MRR@10 and R@10 for MS MARCO dataset.

The main results are shown in Table 2. From the results, we can find that: (1) For the two traditional probabilistic ranking models, we can see that BM25 is a strong baseline which performs better than QL in most cases. (2) For LTR models, RankSVM and LambdaMart perform better than the traditional probabilistic ranking models. It is not surprising since LTR models combine various features including the two traditional probabilistic ranking models (i.e., BM25 and QL scores), which are capable of characterizing the relevance between a document and a query. However, Prank performs worse than the traditional probabilistic ranking models on the Robust04 and MQ2007 dataset. The reason might be that pointwise LTR ignores the fact that some documents are associated with the same query and some others are not (Liu, 2011). (3) Most neural ranking models perform worse than traditional probabilistic ranking models and LTR models. The reason might be that it is difficult for a deep neural model to train from scratch with such a few supervised pairs (Qin et al., 2020). (4) Pre-trained models (i.e., BERT and ColBERT) perform the best on all the three datasets. These results indicate that the rich language information from text captured by the pre-trained model is useful for relevance modeling.

The performance variance comparisons among different ranking models in terms of VNAP are shown in Figure 2. Recall that high values of VNAP attest to decreased ranking robustness.

From the results, we can observe that: (1) For the two traditional probabilistic ranking models, BM25 achieves lower VNAP values than QL on the Robust04 and MQ2007 datasets, indicating that BM25 is more robust than QL on these datasets. (2) For the LTR models, RankSVM and LambdaMart achieve lower VNAP than traditional probabilistic ranking models on the three datasets, indicating that LTR models are more robust than traditional probabilistic ranking models. (3) For neural ranking models, DSSM achieves the highest VNAP on the MS MARCO dataset. The reason might be that DSSM achieves poor mean performance on the MS MARCO dataset and the corresponding VNAP is relatively higher. (4) When we look at the pre-trained models (i.e., BERT and ColBERT), we find they achieve the lowest VNAP among all the ranking models on the three datasets. It is interesting to find that pre-trained models exhibit the strongest robustness in terms of VNAP. As shown in Table 2, the pre-trained models have shown great success on the average effectiveness. The result indicates that pre-trained models present a great potential since it performs well in terms of the mean performance and the robustness (e.g, in terms of the variance).

Here, we analyze the relationship between the average effectiveness and the robustness. Specifically, we consider the variance of effectiveness in Figure 2 and the average effectiveness in Table 2 simultaneously. Overall, the robustness of different models generally increases (i.e., the VNAP decreases) with the increase of the average ranking effectiveness (i.e., the MAP increases). In addition, we have computed the correlation between the effectiveness and the variance. For example, the Pearson correlation coefficient between the MAP and VNAP is $-0.9869$. The result indicates that there is a strong correlation between the average effectiveness and the variance of effectiveness.

To evaluate the robustness of ranking models in terms of the poorly-performing queries, we use two metrics following the previous works (Voorhees, 2004).

• •

  %no denotes the percentage of queries with no relevant documents in the top 10 retrieved, i.e.,

  (8)

  $$\%no=\frac{1}{|Q|}\sum_{t=1}^{|Q|}\prod_{n=1}^{10}\delta(y_{tn}=0),$$

  where $y_{tn}\in Y$ is the relevance label (i.e., the larger the relevance label, the more relevant the query-document pair) of the top $n\in[1,10]$ document with respect to the query $q_{t}$. $\delta$ is the indicator function and $|Q|$ is the total number of evaluated queries. The ranking model would be more robust with a lower %no value.

• •

  gMAP denotes the geometric mean average precision, which is defined as,

  (9)

  $$gMAP=\text{exp}(\frac{1}{|Q|}\sum_{t=1}^{Q}\text{log}(AP(q_{t})+\epsilon))-\epsilon,\\$$

  where $AP(q_{t})$ is formulated as in Eq. (7). $\epsilon$ is a minimal positive since the average precision score for a single query may approximate to zero (Voorhees, 2006). The ranking model would be more robust with a higher gMAP value.

Here, we first measure the performance of the poorly-performing queries in terms of %no and gMAP. Then, we analyze the relationship between the performance of all the queries and the performance of the poorly-performing queries.

• •

  How does different ranking models perform over poorly-performing queries? We first focus on the poorly-performing queries to analyze the robustness of different ranking models in terms of %no and gMAP. The main results are shown in Table 3. Recall that low values of %no and high values of gMAP attest to increased ranking robustness. From the results, we can observe that: (1) For traditional probabilistic ranking models, QL achieves a lower %no value than BM25 on the MQ2007 and MS MARCO, demonstrating that QL is more robust than BM25 over the poorly-performing queries. (2) For LTR models, RankSVM and LambdaMart achieve lower %no values and higher gMAP values as compared with traditional probabilistic ranking models on the MQ2007 and MS MARCO. The results indicate that combining different kinds of human knowledge (i.e., relevance features) could achieve good improvements on the ranking robustness over the poorly-performing queries. (3) When we look at the pre-trained models (i.e., BERT and ColBERT), we find that BERT achieves the lowest %no value and the highest gMAP value among all the models on the MQ2007 dataset, while ColBERT achieves the lowest %no value and the highest gMAP value on the MS MARCO dataset. It is interesting that pre-trained models exhibit the strongest robustness over the poorly-performing queries under the I.I.D. setting.

  

  

• •

  What is the relationship between the average effectiveness and the robustness over the poorly-performing queries? Specifically, we visualize the experimental results on the MQ2007 dataset from Table 2 and 3 into Figure 3, where the horizontal axis represents the average effectiveness metric (i.e., MAP) and the vertical axis represents the robustness metric (i.e., gMAP and %no). We have the following observations: (1) The robustness over the poorly-performing queries generally increases (i.e., the gMAP increases), with the increase of the average effectiveness (i.e., the MAP increases). The reason might be that the good performance over poorly-performing queries, in turn, would improve the performance over all the queries. This is consistent with the finding in previous studies where the degree of ranking robustness is positively correlated with retrieval performance (Zhou and Croft, 2006). (2) The pre-trained models are the most robust model over the poorly-performing queries and the most effective model over all the queries under the I.I.D. setting. Future work could propose new self-supervised objectives tailored for IR that enhance model robustness.

The OOD generalizability on unforeseen query type refers to the transfer effectiveness on unseen query types. Nowadays, users formulate their queries in the form of various types (Broder, 2002) that can describe their information needs properly to find the right results quickly. When submitting a new query type to a search engine, it tends to fail due to the large gap between normal in-distribution and undesired OOD query types. To improve users’ satisfaction, a ranking model should be robust to such new query types.

Formally, given a ranking model $f$ learned on $\{q_{i},\textbf{d}_{i},\textbf{y}_{i}\}_{i=1}^{m}$ which are drawn from the training distribution $\mathcal{G}_{q}$, we aim to evaluate its performance on the test examples with unforeseen query types, which are drawn from a new distribution $\mathcal{G}^{\prime}_{q}$. Specifically, the OOD generalizability on unforeseen query type is defined as

where $q_{t}^{\prime}$, $\mathbf{d}_{t}$ and $\mathbf{y}_{t}^{\prime}$ denotes the query, the document list and the label with new query types. Note that the same document collection is used for both the training and test sets.

To measure the OOD generalizability of the ranking models on unforeseen query type, we propose one automatic metric, namely,

• •

  DR_OOD evaluates the drop rate between the ranking performance $P_{OOD}$ on the OOD test set (i.e., $(q_{t}^{\prime},\mathbf{d}_{t},\mathbf{y}_{t}^{\prime})\sim\mathcal{G}^{\prime}_{q}$) and the ranking performance $P_{I.I.D.}$ on the I.I.D. test set (i.e., $(q_{t},\mathbf{d}_{t},\mathbf{y}_{t})\sim\mathcal{G}_{q}$), which is defined as

  (11)

  $$DR_{OOD}=\frac{P_{OOD}-P_{I.I.D.}}{P_{I.I.D.}},$$

  where $P_{I.I.D.}$ and $P_{OOD}$ are defined as

  (12)

  $$P_{I.I.D.}=\mathbb{E}_{(q_{t},\mathbf{d}_{t},\mathbf{y}_{t})\sim\mathcal{G}_{q}}M(\pi(q_{t},\mathbf{d}_{t},f),\mathbf{y}_{t}),$$

  (13)

  $$P_{OOD}=\mathbb{E}_{(q_{t}^{\prime},\mathbf{d}_{t},\mathbf{y}_{t}^{\prime})\sim\mathcal{G}^{\prime}_{q}}M(\pi(q_{t}^{\prime},\mathbf{d}_{t},f),\mathbf{y}_{t}^{\prime}),$$

  where the effectiveness evaluation metric $M$ can be defined in different ways, such as mean reciprocal rank (MRR) (Radev et al., 2002), mean average precision (MAP) (Schütze et al., 2008), normalized discounted cumulative gain (NDCG) (Burges et al., 2005) and Precision (P) (Croft et al., 2010), with respect to the specific experimental dataset. The ranking model would be more robust with a higher DR_OOD.

The OOD generalizability on unforeseen corpus refers to the transfer effectiveness on unseen corpus. In practice, the training corpus usually has a limited volume and hardly characterizes the entire distribution. Chasing an evolving data distribution is costly, and even if the training corpus does not become stale, models will still encounter unexpected situations at the test time (Hendrycks et al., 2020). Accordingly, training a ranking model on the given corpus that can well generalize to another new corpus is necessary.

Formally, given a ranking model $f$ learned on $\{q_{i},\textbf{d}_{i},\textbf{y}_{i}\}_{i=1}^{m}$ which are drawn from the training distribution $\mathcal{G}_{d}$, we aim to evaluate its performance on the test examples from unseen corpus, which are drawn from a new distribution $\mathcal{G}^{\prime}_{d}$. Specifically, the OOD generalizability on unforeseen corpus is defined as

where $q_{t}^{\prime}$, $\mathbf{d}_{t}^{\prime}$ and $\mathbf{y}_{t}^{\prime}$ denotes the query, the document list and the label from unseen corpus.

To measure the OOD generalizability on unforeseen corpus, we also employ the DR_OOD metric based on the ranking performance $P_{OOD}$ on the OOD test set (i.e., $(q_{t}^{\prime},\mathbf{d}_{t}^{\prime},\mathbf{y}_{t}^{\prime})\sim\mathcal{G}^{\prime}_{d}$) and the ranking performance $P_{I.I.D.}$ on the I.I.D. test set (i.e., $(q_{t},\mathbf{d}_{t},\mathbf{y}_{t})\sim\mathcal{G}_{d}$).

For evaluation purposes, we build two benchmark dataset based on several existing IR collections. Specifically, we use Robust04, MQ2007, and MS MARCO, which have been described in Section 3.3.1. In these IR collections, queries are associated with the metadata information, which helps differentiate the examples with respect to query type and corpus, respectively. We now describe the detail of the two datasets for evaluating the OOD generalizability on unforeseen query type and corpus as follows.

• •

  Dataset for Unforeseen Query Type. Here, we take the MS MARCO dataset as our source data. The reason is that MS MARCO contains various types of questions, and the amount of questions are much larger than other datasets. Firstly, we leverage the official query types ¹⁰¹⁰10https://github.com/microsoft/MSMARCO-Question-Answering, including Location, Numeric, Person, Description and Entity, to identify the query type for each query. Then, for each query type, we construct the new training/test set, by randomly sampling 15,000/300 queries from the original training/development set respectively. In this way, we obtain a benchmark dataset with 5 different query types. The detailed statistics are shown in Table 4.

  type	Location	Numeric	Person	Description	Entity
  #Training Queries	15,000	15,000	15,000	15,000	15,000
  #Test Queries	300	300	300	300	300
  Avg #Relevant Documents	1	1	1	1	1
  Avg query words	5.68	6.77	5.96	5.56	6.27

• •

  Dataset for Unforeseen Corpus. We take Robust04 from news articles, MQ2007 from Gov2 documents, and MS MARCO from web documents, as the whole benchmark dataset to mimic the different data distributions in different corpora. As we can see, they represent different sizes and genres of heterogeneous text collections. For Robust04 and MQ2007, we randomly divide them into a training set (80%) and a test set (20%). For MS MARCO, to obtain a similar volume of relevant query-document pairs for fair comparison, we build the new training set by sampling a quarter of queries from the original training set and directly leverage the original development set as the new test set following (Cohen et al., 2018). For three corpora, we also randomly sample 20% queries from the training set for validation, respectively. Thus, we can obtain a benchmark dataset with 3 different corpora.

The implementation details, including the pre-processing and model implementation, are similar to that in Section 3.3.2. The only expectation is that the parameters of traditional probabilistic ranking models are tuned on the corresponding development set in the two constructed benchmark dataset.

Firstly, we give an overall performance analysis on all the ranking models. We can observe that: (1) Under the I.I.D. setting (e.g., the training and test query type is Person), ColBERT generally performs the best (0.398 on Person) followed by LambdaMART (0.313 on Person), BERT (0.277 on Person) and then RankSVM (0.264 on Person), in terms of MRR@100. (2) As show in Figure 4, most ranking models are not able to well generalize to OOD query types. Take the most effective model ColBERT as an example, the DR_OOD value is -30.2% when the training query type is Location and the test query type is Numeric. It indicates that higher effectiveness does not reliably improve OOD generalizability. (3) When being trained on four query types and tested on the one remaining type, most ranking models perform worse than the I.I.D. setting, while perform better than being trained on the single query type. It indicates that the training data for a single query type may be inadequate. When the training query type increases, the OOD generalizability improves.

When we look at the traditional probabilistic ranking models and LTR models, we find that: (1) In general, traditional probabilistic ranking models achieve the highest DR_OOD values, indicating that traditional probabilistic ranking models are the most robust models when facing unforeseen query types. A possible reason would be that as unsupervised methods, traditional probabilistic ranking models avoid the problem of overfitting the training data and thus have a better OOD generalizability on unforeseen query types. (2) The DR_OOD values of LTR models are higher than that of neural ranking models, indicating that LTR models are more robust than neural ranking models. The reason might be that hand-crafted features in LTR models could better generalize to unforeseen query type than automatically learned features by neural networks.

When we look at the neural ranking models, we find that: (1) In general, neural ranking models achieve the lowest DR_OOD values among all the ranking models. The reason might be that neural ranking models with a deeper network architecture fit the normal in-distribution query types better, at the cost of further loss in performance on the held out OOD query types. It is consistent with the finding in (Cohen et al., 2018). (2) Among these five neural ranking models, pre-trained models (i.e., BERT and ColBERT) have the best OOD generalizability on unforeseen query type. For example, ColBERT trained (fine-tuned) on Person and tested on Location, the MRR@100 value even improves 5.6% compared with ColBERT trained and tested on Location. A possible explanation would be that pre-training on the huge text corpus can improve OOD generalization. It is consistent with the finding in (Hendrycks et al., 2020), which indicates that pre-trained models are more robust to OOD examples on several NLP tasks. (3) Pre-trained models have shown great effectiveness under both the OOD and I.I.D. settings, and good robustness to OOD query types. Future works could design novel pre-training objectives tailored for IR that enhance ranking robustness.

Firstly, we give an overall analysis on all the ranking models. We can observe that: (1) Under the I.I.D. setting (e.g., the training and test corpus is MQ2007), ColBERT generally performs the best (0.4759) followed by BERT (0.4656) and RankSVM (0.4601), in terms of MAP. (2) As show in Figure 5, almost all DR_OOD values are negative. It indicates that most ranking models are not able to well generalize to OOD examples. (3) Besides, the DR_OOD value with respect to MRR for unforeseen MS MARCO is much lower than that for unforeseen query types sampled from MS MARCO. A possible reason is that compared with different query types sampled from the same corpus, different corpora have greater differences among samples.

When we look at the traditional probabilistic ranking models and LTR models, we can find that: (1) Under the I.I.D. setting, BM25 performs better than QL on most corpus, and BM25 is a strong baseline which performs better than LTR models on Robust04. Under the OOD setting, QL is more robust to the unforeseen corpus than BM25. (2) LTR models show a good transfer effectiveness from MS MARCO to Robust04. For instance, the DR_OOD value based on MAP is close to $0$ (i.e., -1.1%) when we train Prank on MS MARCO and then test it on Robust04. Besides, it is surprising that when we train RankSVM on MS MARCO and test it on Robust04, the DR_OOD value based on P@20 is positive (i.e., 2.7%). The reason might be that the distribution of hand-crafted features for Robust04 is similar to that for MS MARCO. (3) Prank shows a better OOD generalizability on unforeseen corpus than RankSVM and LambdaMART if we test them on Robust04 and MQ2007. For example, when we train Prank on MQ2007 and test it on Robust04, the DR_OOD value based on NDCG@20 is even positive (i.e., 8.6%), while LambdaMART achieves -13.2% DR_OOD under the same setting. One possible explanation is that Prank has simpler model architecture and less parameters compared with RankSVM and LambdaMART, which alleviates the problem of overfitting and increases the OOD generalizability.

When we look at the neural ranking models, we can see that: (1) In general, neural ranking models show the worst OOD generalizability on unforeseen corpus. The finding is consistent with that on unforeseen query type. For example, when we train Conv-KNRM on MQ2007 and test it on MS MARCO, the DR_OOD value based on MRR@100 is even -83.8%. (2) DRMM shows a good transfer effectiveness from Robust04 to both the MS MARCO and MQ2007. As shown in previous studies (Guo et al., 2016), DRMM have performed quite well on Robust04. In this way, DRMM trained on Robust04 may lean to generalize to other corpora.

The defensive ability against query attack refers to that the ranking performance on the attacked queries. In a real-world scenario, the users’ queries may be attacked unconsciously. For example, users may occasionally misspell or mistype a query keyword when performing a search. Such query typo is a longstanding real-world problem for IR, which has been extensively studied (Hagen et al., 2017; Zhuang and Zuccon, 2021; Penha et al., 2021). From the perspective of a user, if the search engine cannot tolerate the query typo, the user’s satisfaction will remarkably drop. A reliable system that always produces acceptable retrieval performance, is more preferred than another system that fails on the occasional query typo. Specifically, we consider the effects of misspelled or mistyped queries by exploring several types of character-level and word-level edits.

Formally, given a ranking model $f$ learned on $\{q_{i},\textbf{d}_{i},\textbf{y}_{i}\}_{i=1}^{m}$ which are drawn from the training distribution $\mathcal{G}$, we aim to evaluate its performance on the I.I.D. test queries $q_{t}$ attacked by a query attack function $a_{q}$, i.e., $a_{q}(q_{t})$. Specifically, the defensive ability against query attack is defined as

where $q_{t},\mathbf{d}_{t}$ and $\mathbf{y}_{t}$ denotes the query, the document list and the label from the I.I.D. test set, respectively.

As for evaluation measure, we employ one automatic metric, namely,

• •

  DR_query evaluates the drop rate between the model performance $P_{o}$ on the original queries (i.e., $q_{t}$) and the ranking performance $P_{t}$ on the attacked queries (i.e., $a_{q}(q_{t})$), i.e.,

  (16)

  $$DR_{query}=\frac{P_{t}-P_{o}}{P_{o}},$$

  where $P_{o}$ and $P_{t}$ are defined as

  (17)

  $$P_{o}=\mathbb{E}_{(q_{t},\mathbf{d}_{t},\mathbf{y}_{t})\sim\mathcal{G}}M(\pi(q_{t},\mathbf{d}_{t},f),\mathbf{y}_{t}),$$

  (18)

  $$P_{t}=\mathbb{E}_{(q_{t},\mathbf{d}_{t},\mathbf{y}_{t})\sim\mathcal{G}}M(\pi(a_{q}(q_{t}),\mathbf{d}_{t},f),\mathbf{y}_{t}),$$

  where the effectiveness evaluation metric $M$ can be defined in different ways, such as MRR@100, with respect to the specific experimental dataset. The ranking model would be more robust with a higher DR_query.

The defensive ability against document manipulation refers to that the ranking performance on the adversarial documents. In the Web search setting, many document authors are “ranking-incentivized” (Goren, 2019). That is, they may introduce modifications to their documents, hoping to rank them higher for some queries by search engines. In this way, these modifications could be almost indiscernible by users and users will loose faith in the search system if they observe these rapid ranking changes. Thus, a ranking model should be robust to such manipulations for maintaining high-quality search results.

Formally, given a ranking model $f$ learned on $\{q_{i},\textbf{d}_{i},\textbf{y}_{i}\}_{i=1}^{m}$ which are drawn from the training distribution $\mathcal{G}$, we aim to evaluate its performance on the I.I.D. test queries $q_{t}$ with the corresponding document list $\textbf{d}_{t}$ attacked based on a document attack function $a_{d}$, i.e., $a_{d}(\textbf{d}_{t})$. Specifically, the defensive ability against document manipulation is defined as

where $q_{t},\mathbf{d}_{t}$ and $\mathbf{y}_{t}$ denotes the query, the document list and the label from the I.I.D. test set, respectively.

As for evaluation measure, we employ Top Change (TC) and Kendall’s-$\tau$ distance (KT) following the previous work (Goren et al., 2018).

Suppose that $L^{(q_{t})}$ is a ranked document list with respect to a given query $q_{t}$ achieved by a ranking model. After the document manipulation on the document list $\textbf{d}_{t}$, a new ranked list $L^{(q_{t})\prime}$ is obtained by the ranking model. The key idea to quantify robustness is to measure the distance between $L^{(q_{t})}$ and $L^{(q_{t})\prime}$. Specifically, the lower the TC and KT value, the lower the “distance” between the two lists, i.e., the more robust the ranking model is.

• •

  TC indicates the similarity between the two ranked lists based on the highest ranked document, i.e.,

  (20)

  $$TC=\mathbb{E}_{(q_{t},\mathbf{d}_{t},\mathbf{y_{t}})\sim\mathcal{G}}\delta(L^{(q_{t})}_{1}\neq L^{(q_{t})\prime}_{1}),$$

  where $L^{(q_{t})}_{1}$ and $L^{(q_{t})\prime}_{1}$ denotes the highest ranked document in the ranked list $L^{(q_{t})}$ and $L^{(q_{t})\prime}$, respectively. $\delta$ is the indicator function, where the value is 1 if the highest ranked document in $L^{(q_{t})}$ and $L^{(q_{t})\prime}$ is different, and 0 otherwise.

• •

  KT is defined as the number of discordant pairs between two paired lists normalized with respect to the number of pairs of items in a list, i.e.,

  (21)

  $$KT=\mathbb{E}_{(q_{t},\mathbf{d}_{t},\mathbf{y_{t}})\sim\mathcal{G}}\frac{\sum_{i,j\in P}\overline{K}_{ij}(L^{(q_{t})},L^{(q_{t})\prime})}{n_{q_{t}}(n_{q_{t}}-1)/2},$$

  where $P$ is the set of unordered pairs of distinct documents in $L^{(q_{t})}$ and $L^{(q_{t})\prime}$. $n_{q_{t}}$ is the size of the ranked list. $\overline{K}_{ij}(L^{(q_{t})},L^{(q_{t})\prime})$ denotes whether the document pair $(d_{ti},d_{tj})$ is a discordant pair. A discordant pair is two documents whose relative ranking in one list is different than that in the other list. Specifically, $\overline{K}_{ij}(L^{(q_{t})},L^{(q_{t})\prime})$ is defined as,

  (22)

  $$\overline{K}_{ij}(L^{(q_{t})},L^{(q_{t})\prime})=\delta((L^{(q_{t})}(d_{ti})-L^{(q_{t})}(d_{tj}))(L^{(q_{t})\prime}(d_{ti})-L^{(q_{t})\prime}(d_{tj}))<0),$$

  where $L^{(q_{t})}(d_{ti})$ and $L^{(q_{t})\prime}(d_{ti})$ are the rankings of the document $d_{ti}$ in $L^{(q)}$ and $L^{(q)\prime}$, respectively.

The datasets used for evaluating the defensive ability are as follows.

To evaluate the defensive ability against query attack, we conduct experiments on the MS MARCO dataset, which has been described in Section 4. We construct the novel training set by randomly sampling a quarter of queries from the original training set. We randomly sample 20% queries from the novel training set for validation. Besides, we directly use the original development set as the novel test set.

To evaluate the defensive ability against document manipulation, we follow the previous work (Goren et al., 2018) to conduct experiments on the ASRC and ClueWeb09-B dataset. The detailed statistics of these datasets are shown in Table 7.

• •

  ASRC. Adversarial Search Collection (ASRC)¹¹¹¹11https://github.com/asrcdataset/ is an adversarial ranking dataset, which aims to analyze content-based ranking competitions so as to shed light on the strategic behavior of publishers (Raifer et al., 2017). The competition included 31 different repeated matches, each of which was with respect to a different TREC’s ClueWeb09 query. The competition was run for eight rounds. Students are incentivized by course-grade rewards to manipulate their documents, in order to have them ranked higher in the next round.

• •

  ClueWeb09-B. ClueWeb09-B is a large Web collection with 150 queries and over 50M English documents, whose queries are accumulated from TREC Web Track 2009, 2010, and 2011.

The model implementation details are similar to that in Section 4.3.2. In the following, we will introduce our methods to simulate the adversarial query attack and document manipulation.

• •

  Implementation of Query Attack. The MS MARCO dataset is preprocessed in the same way as described in Section 4.3.2. We use the official top 100 ranked documents retrieved by QL. Then, to implement the query attack, we simulate the query attack at the character-level and word-level.

  For character-level query attack, we follow the previous works (Jones et al., 2020; Pruthi et al., 2019), which are inspired by psycholinguistic studies (Rawlinson, 2007; Davis, 2003). The psycholinguistic studies demonstrate that humans can comprehend text altered by jumbling internal characters, provided that the first and last characters of each word remain unperturbed.

  Specifically, we explore to perturb queries with four types of character-level edits: (1) Add, inserting a new lower-case character internally in a word; (2) Remove, deleting an internal character of a word; (3) Substitute: substituting an internal character for any letter; (4) Swap, swapping two adjacent internal characters of a word. For each query in the test set, we firstly randomly choose a word and then attack it using one randomly-chosen character-level edit. We denote such attacked queries as 1-char attacked queries and there are 25.32% Add, 24.96% Remove, 23.34% Substitute and 24.42% Swap in the final obtained 1-char attacked queries.

  Furthermore, we also randomly select one word from the 1-char attacked queries and then attack it using one randomly-chosen character-level edit (Pruthi et al., 2019). We denote such attacked queries as 2-char attacked queries. We evaluate the performance of different ranking models on the original test queries, 1-char and 2-char attacked test queries respectively.

  For word-level query attack, we explore to perturb queries with three types of word-level edits: (1) Add, inserting a new lower-case word in a query; (2) Remove, deleting a word of a query; (3) Substitute: substituting a word with a random word. For each query in the test set, we firstly randomly choose a word and then attack it using one randomly-chosen word-level edit.

• •

  Implementation of Document Manipulation. We use the public dataset ASRC that was created as a result of an on-going ranking competition (Raifer et al., 2017), to measure the document manipulation (Goren et al., 2018). Specifically, the ranking competition involved 31 repeated matches that lasted for 8 rounds, where each match was with respect to a different query. Students in an IR course served as documents’ authors. In the first round, in addition to the query itself, students were provided with an example relevant document, and were incentivized by bonus points to the course’s grade to modify their documents so as to have them ranked as high as possible in the next round. Starting from the second round, students were presented with the ranking as well as the content of all documents submitted in the previous round in the match. To assure the fairness of the ranking competition, students had no prior knowledge of the ranking function and all data was anonymized. In this way, the ASRC dataset could simulate the document manipulation in the ranking competition.

  Since the ASRC dataset is too small to effectively train a ranking model, we firstly train the ranking models on the ClueWeb09-B, and then evaluate their defensive abilities on the ASRC (Goren et al., 2018). Specifically, we randomly sample 75% queries from the ClueWeb09-B for training and leverage the remaining 25% queries for validation. For the ClueWeb09-B dataset, we use the QL model to retrieve the top 100 ranked documents to build an initial document list.

We first analyze the defensive ability against the character-level query attack. In the following, we first given an overall analysis of different ranking models. Then, we analyze traditional probabilistic ranking models, LTR models and neural ranking models, respectively.

• •

  Overall analysis on all the ranking models. Firstly, we given an overall analysis on all the ranking models against the query attack. We can observe that: (1) In the absence of any query attack, ColBERT performs the best (0.3360) followed by BERT (0.2798), LambdaMART (0.2710) and then BM25 (0.2612), in terms of MRR@100. However, even single-character attacks can be catastrophic, resulting in a significantly degraded performance of 26.1%, 27.6%, 13.2% and 23.3% in terms of DR_query, for ColBERT, BERT, LambdaMART, and BM25, respectively. (2) For all the ranking models, the MRR@100 performances on both 1-char and 2-char attacked queries are worse than that on the original queries. These results indicate that all the ranking models are not able to generalize to attacked queries quite well. (3) The absolute drop rate value between 2-char attacked queries and original queries are much higher than that between 1-char attacked queries and original queries (e.g., 46.4% v.s. 20.8% for RankSVM). By conducting further analysis, we find that the absolute drop rate value could be larger with more characters in a query attacked. Therefore, it is necessary to improve the robustness of ranking models against the query attack. (4) Overall, in terms of the DR_query between 1-char attacked and original queries, the relative robustness order of defending against query attack is DRMM ¡ QL ¡ BERT ¡ ColBERT ¡ BM25 ¡ RankSVM ¡ Prank ¡ LambdaMART ¡ Conv-KNRM ¡ Duet ¡ DSSM.

  	Original	1-char		2-char
  	MRR@100	MRR@100	DRquery	MRR@100	DRquery
  QL	0.2221	0.1540-	-30.7%	0.1213-	-45.4%
  BM25	0.2612	0.2004-	-23.3%	0.1453-	-44.4%
  Prank	0.2311	0.1839-	-20.4%	0.1218-	-47.3%
  RankSVM	0.2535	0.2008-	-20.8%	0.1360-	-46.4%
  LambdaMART	0.2710	0.2353-	-13.2%	0.2100-	-22.5%
  DSSM	0.1220	0.1210	-0.8%	0.1209	-0.9%
  DRMM	0.1335	0.0875-	-34.5%	0.0828-	-38.0%
  Conv-KNRM	0.1716	0.1526-	-11.1%	0.1359-	-20.8%
  Duet	0.1392	0.1315	-5.5%	0.1310	-5.9%
  BERT	0.2798	0.2026-	-27.6%	0.1376-	-50.8%
  ColBERT	0.3360	0.2605-	-26.1%	0.1640-	-51.2%

• •

  Analysis on traditional probabilistic ranking models and LTR models. When we look at the traditional probabilistic ranking models and LTR models, we find that: (1) Traditional probabilistic ranking models are less robust than LTR models under 1-char attack (e.g., -30.7% of QL v.s. -13.2% of LambdaMART). Specifically, the adversarial edits might flip words either to a different word in the vocabulary or, more often, to the out-of-vocabulary token UNK. Consequently, adversarial edits can degrade ranking models by transforming the informative words in a query to UNK. For traditional probabilistic ranking models, they emphasize too much on exact matching signals between the query and the document, and treat each unique character combination differently, resulting in the vulnerability to query attack. (2) For the LTR models, LambdaMART is more robust than RankSVM and Prank, especially against the 2-char attack. One possible explanation is that as a listwise LTR method, LambdaMART can make use of the whole ranked document list, which can better reduce the effect of character-level adversarial attacks with more context information than the pairwise LTR method RankSVM and the pointwise LTR method Prank.

• •

  Analysis on neural ranking models. When we look at the neural ranking models, we find that: (1) DSSM is the most robust model against both the 1-char and 2-char query attack, but performs poorly on the original query set. The reason might be that DSSM utilizes a character-level n-gram based word hashing, which is more robust to misspelling problems than just treating each word as the basic semantic units. (2) Duet is the second most robust model against both the 1-char and 2-char query attack, while also performs poorly on the original query set. Specifically, Duet leverages both the local and distributed representations for text matching. In the distributed representation, an activation pattern that has some errors or other differences from past data can still be mapped to the query, using a similarity function. In this way, Duet is robust to noise and has the ability to generalize (Mitra et al., 2017). (3) Since DSSM and Duet have shown strong robustness, a promising direction to design a robust neural ranking model against the misspelling lies in applying the character-level operations. For instance, we could combine pre-training objectives with the n-gram word hashing layer to simultaneously achieve good ranking effectiveness and robustness. (4) DRMM is the least robust model against the 1-char attacked queries. The result indicates that the adversarial edits significantly degrade the DRMM model which directly uses GloVe (Pennington et al., 2014) word embeddings, by transforming the informative words to UNK. Besides, for DRMM, the improvement of the absolute drop rate value with respect to 2-char queries over 1-char queries is only 3.5%. It is an interesting finding that while DRMM is susceptible to adversarial typos, it shows some robustness against the further attacks. (5) When we take a look at the pre-trained models (i.e., BERT and ColBERT), the phenomenon is absolutely different with DRMM. For example, for ColBERT, the improvement of absolute drop rate value with respect to 2-char queries over 1-char queries is 25.1%, which is the highest among all the ranking models. These results indicate that the pre-trained models are less robust than other ranking models against the query attack. One possible explanation could be that such pre-trained models apply WordPiece (Wu et al., 2016) tokenization, where a word is broken down into more than one sub-words. In this way, an attacked word in a query may be decomposed into more than one attacked sub-words, which have a more significant impact on the performance.

We analyze the defensive ability against the word-level query attack. From the results, we can observe that: (1) Overall, for all the ranking models, the MRR@100 performance on the word-level query attack is worse than that on the original queries. The results indicate that all the ranking models are not able to generalize to word-level attacked queries quite well. (2) In terms of the DR_query between word-level attacked and original queries, the relative robustness order of defending against word-level query attack is QL ¡ ColBERT ¡ DRMM ¡ BERT ¡ BM25 ¡ Prank ¡ RankSVM ¡ Conv-KNRM ¡ LambdaMart ¡ Duet ¡ DSSM. (3) Traditional probabilistic ranking models are less robust than LTR models under word-level attack (e.g., -35.5% of QL v.s. -8.4% of LambdaMART). Meanwhile, for the LTR models, LambdaMART is more robust than RankSVM and Prank. The reason is similar to what we have analyzed in Section 5.4.1. (4) It is surprising to find that DSSM and Duet are the most robust ranking models against the word-level query attack. In addition, the DR_query of Conv-KNRM is also relatively high (e.g., -11.0%). The three ranking models all apply the character-level n-gram operations. The results indicates that ranking models which contains a character-level operation will be more robust than those ranking models which treat each word as the basic semantic units. (5) When we take a look at pre-trained models (i.e., BERT and ColBERT), we could find they are not robust compared with other neural ranking models. For example, the DR_query of ColBERT is -33.9%, which is the highest among all the neural ranking models. One possible explanation could be that WordPiece (Wu et al., 2016) tokenization decompose an attacked word into more attacked sub-words, which have a more significant impact on the performance.

We first analyze the average effectiveness of ranking models on the ASRC dataset. Specifically, we use the mean average precision (MAP) (Schütze et al., 2008) and normalized discounted cumulative gain at rank 5 (NDCG@5) (Burges et al., 2005) to evaluate ranking effectiveness (Goren et al., 2018). The results are shown in Figure 6.

From the results we observe that: (1) All the ranking models can generally achieve good effectiveness on the ASRC dataset. The high MAP and NDCG values can be attributed to the fact that most documents generated by the students were judged to be relevant (e.g., 1113 out of 1279 documents are relevant) (Goren et al., 2018). (2) Overall, traditional probabilistic ranking models perform better than neural ranking models and LTR models. (3) Among all the neural ranking models, DSSM performs the best. (4) BERT performs second only to DSSM in neural ranking models. Specifically, we train the ranking models on the ClueWeb09-B dataset and then evaluate them on the ASRC dataset, i.e., under the OOD setting. As noted in Section 4, we have found that BERT shows greater robustness than other neural ranking models in terms of the OOD generalizability, which is quite consistent with the result on the ASRC. (5) Overall, the relative average effectiveness order in terms of MAP is DRMM ¡ Conv-KNRM ¡ ColBERT ¡ Duet ¡ BERT ¡ LambdaMART ¡ DSSM ¡ Prank ¡ QL = BM25 ¡ RankSVM.

Here, we analyze the robustness of ranking models against document manipulation on the ASRC dataset. The results are shown in Figure 7. Recall that high values of KT and TC attest to decreased ranking robustness, as these are measures of inter-list distance.

From the results we observe that: (1) For the traditional probabilistic ranking models, QL and BM25 achieve the lowest TC and KT performance among all the ranking models, indicating that traditional probabilistic ranking models are the most robust to defend against document manipulations. In the generation process of the ASRC dataset, students were asked to modify relevant documents so as to have them ranked as high as possible in the next round. This suggests that traditional probabilistic ranking models have the lowest sensitivity to the difference between relevant documents. (2) For the LTR models, the ranked lists induced by RankSVM and Prank are more robust than those induced by LambdaMART. Since RankSVM and Prank is linear and LambdaMART is not, the variance of RankSVM and Prank is in general lower, and we saw that its ranking robustness is higher. Meanwhile, the ranked lists induced by Prank is less robust than those induced by RankSVM, indicating that pointwise learning objective may suffer more from the document manipulation compared with pairwise learning objective. (3) For the neural ranking models, they achieve comparable results with LTR models against document manipulation. The reason might be that, flipping original words in a document to new words for higher rank, both the statistical features and word embeddings change significantly, resulting in the performance drop. (4) Overall, the relative robustness order of defending against document manipulation (e.g., in terms of TC) is LambdaMART ¡ Prank ¡ DSSM ¡ BERT = Duet ¡ Conv-KNRM ¡ ColBERT ¡ RankSVM ¡ DRMM ¡ QL ¡ BM25.