Towards a Personal Health Knowledge Graph Framework for Patient Monitoring

The MIMIC-III[5] dataset was used to test the prototype system and validating some components of the proposed framework. MIMIC-III[5] is a large freely accessible database featuring anonymized data on patients admitted to critical care units at a tertiary care hospital. This gave us a multimodal dataset of both structured and unstructured data, including vital sign measurements and clinical notes, for a large number of patients with varying symptom severity. We used this data to test our risk scoring and alert generation for COPD patients, and it demonstrated the challenges of managing such data. Although our system is designed to be as independent of specific data sources as possible.

The proposed framework consists of the following main components (see Figure 1 for the overall architecture): 1) Selection, preparation and merging of ontologies for relevant disease management; 2) Data processing for transformation, annotation of different data streams as well as the associated metadata; 3) Constructing general knowledge graph (KG) from the ontology, adding rules/ risk scoring systems based on the literature and inferences from the ontology’s expert knowledge; 4) Creating nodes (vertices) and relationships (edges) from the transformed data by mapping them to the ontological concepts and storing them in a graph database, e.g. Neo4j; 5) Reasoning/Inferences on the graph data using rules based reasoning or model based inferences; 6) Extract personalised health KGs by retrieving relevant subgraphs, update or augment the PHKGs with new fact or prediction from the reasoning/inference; 7) API for downstream tasks on the PHKGs.

1) Ontologies. For the use case, an ontology was developed to store the expert knowledge semantic terminology relevant to monitoring COPD symptoms and patient condition. The ontology constructed for the project is known as COPH (COpd and Physical Health). This is formed of multiple ontologies, imported and merged in Protege[6], that have different discrete domain knowledge (ECG, ICD9 codes, diseases etc.) that are collectively relevant to perceived use cases for the platform as whole. This ontology contains 106,242 classes considered relevant for managing COPD and the associated physical health context. Snomed-CT ontology is also included for annotation of free text.

2) Data harmonization.
Time series data summarization. This is an important step for constructing temporal KGs. For example, in our use case of MIMIC-III data, the summaries (such as averages and the mode) of the measurements are splitting a day into quarters, i.e., a summary for each 6 hours of the day. Timestamps are stored typically as properties of a node and can be accessed easily through queries, and smart filters can be applied using comparisons with other dates, such as only returning results featuring a timestamp within a time/date-range.

Statistics can also be computed for detecting condition shifts, e.g. the Kolmogorov-Smirnov test can be used to compare the distribution of the most recent 3 days of the patient’s measurements with the rest of the month before it in our use case. This comparison can help to understand whether something concerning may be happening with the patient’s symptoms acutely.

Dealing with free text. Appropriate language models can be utilized for extract and annotate the free text into key clinical concepts and relationships, often model fine-tuning might be needed for the target domain. Here we utilized the Medical Concept Annotation Toolkit (MedCAT), which is an open-source library providing Named Entity Recognition and Linking (NER+L) functionalities, as well as an annotation tool and an online learning training interface [7]. For our use case, we initially extracted information from MIMIC III patient note in order to configure the vocab and concept database (CDB) for the MedCat model.

The vocab model has functions of spell checking and word embedding, which can be pre-calculated using word2vec or BERT. The concept database is a custom built using the chosen ontologies such as Snomed-CT. We then performed self-supervized training on the raw document to train or refine the MedCat models, which will be used for NER+L on the documents. The resulting annotation output would be imported into a graph database and merge with the PHKGs for corresponding patients.

Harmonizing multimodal data. Biomedical research and precision medicine increasingly rely on omics data, such as genomics and metabolomics, as well as medical images and documents. These diverse data types can be harmonized based on semantic summarization of clinical reports or predictions from multimodal biomedical AI models. Multimodal analysis can generate actionable insights, such as disease subtypes, biomarker profiling, and associated pathways [8]. These results can be annotated to the relevant concepts and then stored and queried conveniently using PHKGs. Relevant pretrained machine learning models can also generate embeddings for data items of different modalities. These embeddings can be stored in the PHKG, ready for graph indexing or inferences based on semantic similarity between nodes or subgraphs.

3) General KG. General KG can be constructed directly from ontologies: we create the instances for the classes from those Internationalized Resource Identifiers (IRIs) featured in the ontology and create relationships between the measurements and the patient (who have their own instances in the knowledge graph). These classes, known as nodes for the knowledge graph, and relationships can then both be written to the knowledge graph.

Additionally, scoring systems for determining risk from measurements or rule-based models for risk assessment can be implemented based on standard healthcare guidelines. These models are explainable, which can help to both trust the insights and understand where refinement and personalization can be added. The KG can be updated based on domain knowledge extracted from ontologies and the literature (including existing knowledge bases) [4].

4) Personal health KG. To integrate the transformed data with the knowledge graph, a semantic mapping layer was implemented to annotate the data using metadata referencing ontological terms. Each class will have an appropriate object property with a theme of possession. For example, a class for “Condition” (diagnosis) would have a corresponding object property “hasCondition” that will be used for the mapping. Appropriate node properties (such as a patient node having an age) would be a property of the node rather than another relationship. This is to keep a clean and efficient architecture that is easier to traverse.

We use MongoDB to store the mapping with version control to support traceability and provide transparency for the nature of the terms and reduce definition confusion. The semantic mapping is part of our transformation layer and the resulting mapped data is ready for storage in the knowledge graph.

5) Reasoning and Inference. A reasoning engine was implemented that can utilize both rule-based and model-based inference. Once we have the terms created for the data and the relationships in place, we can consider the terms to see what can be inferred from the data. Relationships such as subclass or ISA are commonly used in queries without explicitly listing associated subclasses in the query statement.

6) Personalization. In the system for COPD, risk scores and higher level vital sign context were inferable through rules set into the graph database based on a combination of the ontology’s expert knowledge and personalized thresholds. Cypher query triggers were set up to trigger upon committing the data to the knowledge graph. The knowledge graph can store both generalized and personalized thresholds and flags for comparison, which can then be updated for refining the personalized care for the patient. A model-based approach can also be utilized for inferring relationships, context or determining patient risk. These scores are evaluated to create alerts of varying grades to indicate severity and levels of urgency to the healthcare provider and patient for either seeking consultation or the provision of appropriate care.

7) Data Storage. A major element of the data management system for the COPD use case, is being able to store the data that will be mapped to the ontology for further processing. MongoDB was chosen for its schemaless, flexible document-store which allows a more relaxed management approach, where both the raw multimodal data, all relevant metadata for data transformation and mapping would be stored.

While this project is using MongoDB specifically for the raw data storage, the architecture as a whole can easily support an alternative database system and is designed to be agnostic to the specific tools, providing that the functionality provides the general inputs and outputs required for the previous and next components in the framework.

Neo4j’s maturity compared to the alternatives made it an easy choice for working into the system for storing PHKGs without being concerned about potential complications, and it has plugins available for the features that may be missing. With timestamp attached to the PHKGs, and a snapshot or a subgraph of the PHKG at a specific timestamp can be easily extracted from the graph database. For time-series data streams, only summary or compressed data would be stored as graphs in order to keep the graph database leaner for faster queries, while also retaining the appropriate context of the patient’s condition.

A separate storage for the rest of the data, such as the MongoDB solution, makes more sense for the simple document structure that will be more memory efficient on larger datasets. Therefore there is a complementary relationship between the database solutions for supporting the framework overall.