Adversarial Attacks Against Deep Learning Systems for ICD-9 Code Assignment

The International Classification of Diseases (ICD) establishes a standardized fine-grained classification system for a broad range of diseases, disorders, injuries, symptoms, and other related health conditions [3]. It is primarily intended for use by healthcare workers, policymakers, insurers and national health program managers. The United States incurs administrative costs in billions of dollars annually arising from a complex billing infrastructure [16]. Specifically, the ICD code assignment is typically a manual process, consuming on average between 25 to 43 minutes per patient depending on the ICD version [1]. It is also prone to errors resulting from inexperienced coders, variation between coders, incorrect grouping of codes or mistakes in the patient discharge summaries. These errors are very costly with one report estimating that preventable errors in ICD coding have cost Medicare system 31.6 billion in FY2018 [2].

Recent work [13, 15, 4] has tried to automate the task of ICD code assignment using deep learning. Typically framed as a multilabel classification problem, researchers have trained Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Transformer models to predict ICD-9 codes from patient discharge summaries. These models have outperformed rule-based approaches and those utilizing conventional algorithms such as Logistic Regression, Support Vector Machines, Random Forests etc., achieving competitive micro F1-scores in the range 42% - 68%. Amongst these models, those based on CNNs have achieved the best performance.

Neural network models have revolutionized the field of NLP and SOTA models for various NLP tasks involve deep neural network models such as BERT, Bidirectional RNN or CNN-based methods. Recent works [10, 11, 14, 19] have shown a particular vulnerability of such deep models to adversarial examples that are often produced by adding small and imperceptible perturbations to the input data. The state of the art models of NLP are no exceptions to such perturbations. [18] provides a review of different adversarial attacks and defense strategies in the NLP literature. Based on granularity of the perturbation, adversarial attack strategies in NLP can be classified into three types - character-level attacks, word-level attacks and sentence-level attacks. In a character-level attack strategy, the model induces noise at the character level. Character-level noise can be induced due to naturally occurring reasons such as typos and misspellings or due to intentional modification by a malicious third-party. [12, 6, 5] are some of the existing character-level attack strategies in NLP. To accurately model the naturally occurring typos, [17] restrict the typos distribution based on the character constraints found in a standard English keyboard. We follow this strategy in our work. Furthermore, we assume a white-box setting where the adversary has access to gradients of the loss function wrt to the model inputs. To our knowledge, this is the first work to investigate the effects of adversarial samples in clinical NLP domain.

We used MIMIC-III [8], a large open source database comprising information of patients admitted to critical care units of Beth Israel Deaconess Medical Center (Boston, Massachusetts, USA). The database contains de-identified Electronic health records with both structured and unstructured data including diagnostics and laboratory results, medications, and discharge summaries. In this work, we focus on discharge summaries which encapsulates details pertaining to a patient’s stay.

Each discharge summary is manually annotated by human coders with multiple ICD-9 codes, describing both the diagnoses and procedures that the patient underwent. Out of the approx. 13000 possible ICD-9 codes, 8921 (6918 diagnosis, 2003 procedure) are present in our dataset. Following previous work, we merge discharge summaries corresponding to the same patient ID, such that no patient appears twice in our dataset resulting in 47,427 discharge summaries. This is done to ensure that there is no ‘data leakage’ between train, validation, and test sets.

The full label setting is quite noisy and suffers from class imbalance. Potential sources of noise include both missed assignments (not annotating all relevant ICD-9 codes) and incorrect assignments (annotating similar but incorrect ICD-9 codes). Consequently, it is relatively trivial to develop an adversarial attack strategy in the full label setting. For instance, one could simply find the keywords corresponding to low frequency labels and then either append or remove them from a discharge summary to alter a machine learning model’s prediction. This strategy will however fail for frequent labels since we expect the model to generalize beyond simply memorizing a few keywords. Therefore, we limited the label set to the 50 most frequent labels and removed discharge summaries which were not annotated with at least one of the labels. The resulting dataset was then split into training, validation and testing sets which contained 8067, 1574, and 1730 discharge summaries, respectively.

We followed the same pre-processing steps as in previous work [13]. All tokens without any alphabetic characters were removed. We then lowercased all tokens and replaced those appearing less than three times in the training documents with an ‘UNK’ token.

Our baseline models were the same as [13]. Specifically, we used a CNN-based sentence classifier model introduced by [9] which utilizes a max pooling layer to get sentence vector representations. We call this model Max Pool based CNN. The other model that we use instead utilizes label embeddings to calculate attention weights over word positions. These weights are then used to pool the output of the convolutional layer and calculate the sentence vector representation. This model is referred to as the Attention Pool based CNN.

We generate adversarial examples based on the following algorithm: Given a pre-trained NLP algorithm $f:X\to y$ and a measure of classification $q:y\to s$, we are interested in finding perturbations $\delta x$ on the input $X$ such that $q(X+\delta x)\leq q(X)$ under the constraint $||\delta x||\leq K$. The final constraint ensures that the perturbations are small. In our work, we consider perturbations (typos) of four types:

1. 1.

   Insert - Insert characters into a word, such as hike $\to$ hlike

2. 2.

   Delete - Delete characters in a word, such as hike $\to$ hke

3. 3.

   Swap - swap two characters of a wors, such as hike $\to$ hkie

4. 4.

   Replace - Replace a character in a word with any neighboring keys in the keyboard, such as hike $\to$ hoke. Here o is a neighboring word to i in a standard english keyboard.

Given an input sentence $s$ that is tokenized according to the model’s tokenizer as $s=(w_{1},w_{2},..,w_{N})$, we compute the partial derivative of loss with respect to each input item as shown below,

Based on this gradient information, we select a input word $w_{i}$ to attack. We experiment with two different strategies here, the maximum gradient strategy where we choose the word corresponding to the maximum gradient and a random strategy where a random word is chosen to attack. Once a word is chosen, we generate all possible typos based on the four ways described above. The typo which decreases the score of the output $y$ based on the score function $q$ is chosen. Here, we use the top5 precision as the score function. Now the word replaced with the optimal typo word is again fed through this loop for $K$ times. Each time, a different word is chosen to ensure that final words don’t change from the initial words by a lot. We experiment with different choices of $K$. The algorithm is shown in alg. 1

To the best of our knowledge, [13] is the current state-of-the-art for the task of automated ICD-9 code assignment. We re-implemented their best performing models using the AllenNLP framework [7]. The test-set performance of the models for the task of predicting the top-50 most frequent ICD-9 codes from discharge summaries is given in table  1. We found that the Max Pool based CNN outperformed the Attention Pool based CNN on all performance metrics. Further, we found that the computation time for training as well as generating predictions for the former was much lesser than the latter. Therefore, we decided to limit our focus on developing an adversarial attack strategy for the Max Pool based CNN.

We experiment with three different values of budget $K=\{10,20,30\}$ and two different strategies - maximum gradient and random strategy for selecting the token to attack. The maximum gradient strategy can be used to analyze the robustness of the model to malicious attacks while the random strategy can be used to simulate natural settings with adversarial examples. The training time for each run on the entire corpus ($1725$ discharge summaries) - $8$ hrs to $16$ hrs on a machine with Tesla K80 GPU. The results are summarized in table  2.

In accordance with our intuition, max grad strategy performs better than random strategy. This is because, max grad strategy can produce meaningful perturbations in a large input space (average size of input document is $\sim 1400$ tokens). The model’s performance doesn’t drop much with random strategy. This suggests that the model is some what robust to naturally occurring noise such as typos and misspellings. However, this might change as the budget is increased. Due to computational limits, we did not explore budgets beyond $30$. A key result of our work is that, with less than $3\%$ of input tokens modified, the model’s performance drops significantly from $0.62$ to $0.377$. This shows the potential vulnerability of this model to malicious attacks. Since, only a very few tokens are changed, it might be hard to defend against these attacks by training a discriminator to distinguish maliciously modified documents from regular ones.

Tables  3 and  4 show examples of discharge summaries before and after attack with their top5 labels. It is important to note that, on a few discharge summaries (last example in both the tables), the algorithm increases the top5 precision instead of decreasing it. One can make modifications to the algorithm to ensure that this doesn’t happen which would result in further drop in precision. Due to time constraints, we were not able to accommodate this modification. Nevertheless, these examples show the brittleness of the baseline model to input tokens.

This work is a first step at exploring the robustness of NLP models used for automatic ICD-9 code classification. Clinical documents are different from regular documents as they are typically generated in a fast-paced environment with higher than average typos and non-standard acronyms. As a result, clinical NLP models are more susceptible to adversarial samples compared to a regular NLP model trained on a standard English dataset. A key extension of the work would be to consider a dictionary learnt from clinical documents and biomedical literature as a defense against these character-level perturbations. Although this might mitigate the decrease in performance, it wouldn’t completely solve it. A more rigorous way to deal with this would be to account for this in the tokenization strategy. It is easy to push a word out of vocabulary when using tokenization strategies like word2vec and GloVe. Other strategies that model words unseen in training dataset such as word-piece and byte-pair encoding will also break when typos are introduced because these models learn sub words from a standard dictionary. Therefore, any defense must account for these typos in the fundamental tokenization strategy. An interesting direction would be to learn a word similarity metric and map an unknown word to a closer word in the vocabulary given the input word and the context in which it appears. Building a robust tokenization strategy would be the first step towards a robust NLP model against character-level adversarial attacks.