Predicting Anti-microbial Resistance using Large Language Models

The genes for antibiotic resistance have increased rapidly over the past 10 years and have become a threat to human health Zhang et al. (2022). Moreover, dangerous infectious diseases like COVID-19 can also spread. In such times, it is important to classify the DNA sequences of antibiotic resistance genes. In bioinformatics, the main method for classifying DNA sequences has been to find similar sequences by aligning two DNA sequences using text alignment Bonin et al. (2023). Recently, there have been methods that use language models created from the nucleotide or protein sequences of various species and fine-tune them to create classifiersBrandes et al. (2022); Ji et al. (2021); Zhou et al. (2023). These methods have the advantage of being able to identify which parts of the nucleotide sequence are important. To fine-tune, databases containing information on antibiotic resistance genes must be used. The main databases are CARD Jia et al. (2017) and MEGARes Doster et al. (2020). Existing methods use the labels associated with antibiotic resistance genes, such as the class to which the resistance gene belongs, for example, the label of the antibiotic to which resistance is present. It is a prediction of a single label from a single gene sequence Kang et al. (2022). However, if we look at the CARD or MEGARes databases, there are several attributes that describe a particular gene. There are Gene Family and Resistance Mechanism. If we use this information when predicting the antibiotic to which resistance is present, it could be helpful for prediction. Here, we get an idea and propose a model that uses human-readable information to predict antibiotic resistance genes. We also provide a method to merge the different classification systems of CARD and MEGARes. We will also explain the LLM-based data augmentation technique for rare classes with few samples.

Our approaches include fine-tuning a pre-trained language model with various species’ gene nucleotide sequence data to predict antibiotic resistance genes and their classes. We also fine-tune a pre-trained language model trained on a corpus containing diverse papers from the fields of biology and medicine to predict the names of antibiotic resistance gene properties. We provide an effective ensemble model Kumari et al. (2021) using the above two models in a weighted soft voting method. To integrate the classes, we combine the DNA sequences and the concepts that describe them from CARD and MEGARes into one. We use the EBI ARO ontology Cook et al. (2016) to combine CARD tagging and MEGARes tagging into one class system. For rare classes with few samples, we use BioGPT Luo et al. (2022) prompting to perform data augmentation.

In all datasets, using a text information-based language model shows a 9.53 accuracy and 30.34 macro f1 score improvement in CARD and 10.38 accuracy and 50.57 macro f1 score improvement in MEGARes. Adjusted ratio ensemble models show better performance compared to other cases through experiments. Existing NT and other nucleotide sequence-based models find it difficult to process natural language. Our fine-tuned text-based language model was trained using a small amount of pre-training resources (40GB A100 GPU). By constructing an ensemble model, it achieves better performance compared to competing models such as AMR-meta Marini et al. (2022), Meta-MARC Lakin et al. (2019), and Deep ARG Arango-Argoty et al. (2018).

To compare with other models, we integrated the class system. This enables comparison with competing models. It also allows us to create models for predicting Gene Family and Resistance Mechanism. In particular, the number of samples corresponding to classes in Gene Family and Resistance Mechanism is very small in many cases. This integration helps to implement Gene Family and Resistance Mechanism prediction models. The integrated class system shows better performance compared to cases where it is not. The number of genes available for training increases.

In some competing models, it is recommended to use reads instead of full genes. In the case of AMR-meta, it aims to predict paired end genes. To compare with these models, it is necessary to generate reads. Reads generation uses ART. ART is a simulator for analyzing nucleotide sequences, and it helps with accurate modeling of biological information data as a software Huang et al. (2012). ART has the advantage of customizable indel error rates Milhaven and Pfeifer (2023). The learning and experiments using these reads are presented in Table . In this experiment, the proposed model also demonstrates strong competitiveness.

AMR-meta is a method for classifying antibiotic resistance in high-speed metagenomic data. This method uses a sequence alignment-free approach based on k-mers and meta-features, and it utilizes both resistant and non-resistant genes as training data. As a result, AMR-meta can more accurately identify antibiotic resistance genes and reduce false-positive rates for non-resistant genes. However, it uses a complex matrix decomposition method to generate meta-features, which can be computationally intensive. Additionally, the prediction performance of AMR-meta may vary depending on the type of antibiotic used or the diversity of the resistance genes. These characteristics make AMR-meta useful for analyzing high-speed metagenomic data, but at the same time, they suggest that it may be limited in certain situations.
AMR++ is a customized bioinformatics pipeline that uses high-throughput sequencing data to predict the diversity and abundance of antibiotic resistance genes (ARGs). This pipeline is integrated with the MEGARes database, allowing for efficient analysis of ARGs in large-scale metagenomic sequencing data. The main advantage of AMR++ is its high throughput and efficiency, enabling users to quickly and accurately analyze complex datasets. In addition, this software can distinguish between types of ARGs, including cases where resistance genes require specific mutations. However, this pipeline requires high-quality assembled and/or translated data, which may cause difficulties or limitations in generating metagenomic datasets. Furthermore, AMR++ may require advanced bioinformatics skills and resources, potentially limiting accessibility for some researchers.
Meta-MARC is a machine learning classifier developed to enhance the detection and classification of antibiotic resistance genes. This system is based on the MEGARes database and uses DNA-based hierarchical Hidden Markov Models (HMMs) to classify antibiotic resistance genes in high-throughput sequencing data. Meta-MARC is robust against various gene mutations, which is particularly useful for non-standard databases and sequences. This tool provides high sensitivity and specificity, playing a crucial role in accurate antibiotic resistance detection. However, Meta-MARC is computationally demanding, particularly when dealing with large datasets, which can result in increased processing time and memory usage. Additionally, high sensitivity settings may potentially increase false positives, so users must carefully interpret the results.
DeepARG is a deep learning-based system used for predicting antibiotic resistance genes (ARGs) in metagenomic data. It utilizes two models, DeepARG-SS and DeepARG-LS, for classifying short and full-length gene sequences. Compared to the traditional ’best hit’ approach, it has the advantage of identifying a wider range of ARG diversity with lower false negative rates. However, the performance of this system heavily depends on the quality of the training database, and it has limitations when it comes to predicting new categories of ARGs. Despite these limitations, DeepARG is a useful tool for evaluating the presence and diversity of ARGs in environmental samples.

As far as we know, our work is the first to combine natural language models and biological sequence models to predict antibiotic resistance genes. We proposed a model that combines two different attribute language models into an ensemble. By using both nucleotide sequence information and its description, including Gene family and resistance mechanism information, it enables more accurate drug class predictions. We also integrated various databases using the EBI ontology and used a large language model (LLM) for data augmentation in classes with insufficient data. As a result, we achieved performance close to the state-of-the-art. We believe this fusion has significant meaning. Moreover, we tested the structure we trained using only nucleotide sequences and obtained acceptable results. This seems promising for future research.

This work is supported by the National Science Foundation.