Large-Scale Data Extraction From the OPTN Organ Donor Documents

We introduce a multi-faceted, interdisciplinary problem with paramount importance to millions of people worldwide, and especially to $\approx 550,000$ Americans with end-stage kidney disease (ESKD) who require dialysis. We aim to address the inefficiencies in organ quality assessment by making previously inaccessible data analyzable through large-scale curation. The symptom of under-par performance of the system is gradual increase in the number of organs that are not used (discarded), e.g., nearly 30% of recovered kidneys are not transplanted, gradually increasing from 8% in the 1980s. If the non-use ratio is reduced, potentially thousands of lives would be saved in the U.S. alone, and dramatically improve life quality for most patients who spent 12 hours a day undergoing hemodialysis which results from a successful kidney transplant. The total cost of dialysis in the US is over $30 billion ($100,000 per patient per year) and simple cost analysis shows that a kidney transplant reduces this to a small fraction, potentially a 3-fold cost reduction per patient. The interdisciplinary research and software here described in this paper target the inefficiencies flow of information in the U.S. organ transplant system, where the primary vehicle for communications is a collection of PDF documents (“PDF attachments”) associated with every donated organ.

The interdisciplinary research and software here described in this paper target the inefficiencies flow of information in the US organ transplant system, where the primary vehicle for communications is a collection of PDF documents (“PDF attachments”) associated with every donated organ. The subject of investigation by our interdisciplinary group is the record of all US organ transplants since electronic data collection started (2008-2022) acquired from OPTN. For practical reasons in this stage of our project we narrowed the dataset to the 2022 portion (10% of the data), which is representative of the entire data, but much easier to manage in the software development cycle. In the current paper we report the preliminary results obtained from this portion of the data, with future publications planned to cover all aspects of the project.

The PDF attachments of interest in the case of DCD donors include the DCD Flowsheet shown in Figure 1. The main data in this form is a time series representing vital signs of a donor in the agonal phase, to be used as predictor of the organ damage, and ultimately determines the graft survival and durability.

Extraction of the time series from PDF documents consists of complex tasks. The textual data can be extracted from PDF in a straightforward fashion (without OCR), and is 100% accurate (in the absence of software glitches and human error). For this task we use open software written in Java, PDFBox, along with an enhancement of the PDF text extraction PDFLayoutTextStripper. This software extracts formatted text. Unlike the software provided inside PDFBox called PDFTextStripper, the enhanced version preserves the layout of the text on the page, thus allowing us to sufficiently correct page segmentation (e.g., discovery of table boundaries). It should be mentioned that PDF is a rich format, which allows storing data tables in native PDF tables. However, these mechanisms are not used in the PDF attachments. The reason can be traced to the fact that the original document is in the Microsoft Word format (.docx) and the PDF is a product of exporting the document to PDF file format. Multiple tools are used to perform the Word-to-PDF conversion, resulting in PDFs that differ internally and significantly enough to trip up schemes relying upon cutting-edge features of the PDF format. The approximation of the plain text content of the PDF preserving layout turns out to be the lowest common denominator which allows achieving almost all objectives related to data extraction.

Other data, besides the time series, includes a survey with $\approx 20$ questions with answers which are either categorical (often: Yes/No) or factual questions such as dates, person names, and volumes. The data is relatively easy to extract. The main issue is that the lack of standard formatting and many ways to refer to the same categorical variable value.

The PDF attachments and medical forms in general present an interesting case of a restricted domain language. Natural language processing (NLP) and computer science (theory of compilation) has made an extensive use of regular grammars, a relatively simple mechanisms for capturing syntax of relatively simple languages. Our method of parsing the documents in question is based on the premise that medical forms can be accurately parsed by regular grammars.

In large problems, such as ours, we are dealing with dozens of medical form types, with many variations amongst them. Therefore, it is important to streamline the process of constructing form-specific regular grammars. In our work to date we constructed them mostly “by-hand”. A rough algorithm is this:

1. 1.

   Extract plain text of a page of a document

2. 2.

   Note fixed text (queries, table headings, etc.)

3. 3.

   Note the data type of the variable portions of the form and construct a regular expression that captures their structure best. This includes numerical data, dates, names, etc.

4. 4.

   Try parsing a representative sample of documents in question

5. 5.

   Identify failures, repeat.

Typically, after several iterations of this process most documents ($>90\%$) are matched and parsed correctly. The byproduct of parsing is assigning semantic values to the patterns representing the variable portions of medical forms, i.e. user input. The process of mapping the obtained values to a database is relatively straightforward.

There are two main ways to represent regular grammars:

1. 1.

   use BNF notation;

2. 2.

   use regular expression.

In applications like ours, the approach based on constructing regular expression appears more productive. As an example, the regular expression in Table 1 matches the text at top of the first page of the DCD flowsheet (“PRE-OPERATIVE MANAGMENT” section). Reading and understanding this page-long regular expression is a daunting task. It should be noted that there are two possible approaches:

1. 1.

   Top down, as described above, where variable portions of the text are replaced with small regular expressions.

2. 2.

   Bottom up approach, where regular expressions are constructed for smaller portions of the text, such as lines of text, and combined

We found the bottom-up approach quicker when building the expressions “by hand”. The term “by hand” does not accurately reflect the process, as in practice we developed a computer program that writes the regular expression in Table 1.

The regular expression, such as the one in Table 1 is of tremendous value because it is a concise and readable (with some practice), description of a language parser. This description allows to complete the task of parsing with one line of code using MATLAB (and other high-level software environments). E.g., in MATLAB the code is this

The system builds a data structure which binds named tokens, such as ’(?<extubated>)’, ’(?<heparine_dosage>)’, etc., to their semantic values set by user when filling out the form, located at the lines 3 and 4 in the JSON output of Table 2.

An example of a form entitled “LIVER DATA” is shown in Figure 3. As we can see, the form is similar to the “PRE-OPERATIVE MANAGMENT” section of the DCD Flowsheet. The notable difference is the presence of 8 checkboxes. To correctly decode this form we must decode the content of the checkboxes, as the status of the checkmark cannot be inferred from the textual information in the form. The method we outlined for the DCD Flowsheets works well here and the block of checkboxes can be seen in Figure 3(c). It should be noted that MATLAB software scans objects left-to-right and then top-down, resulting in the specific ordering of checkboxes. It should be noted that although checkboxes in the PDF file are “perfect” and resolution-independent in the PDF file, they must be rendered (in this case, the resolution is 300dpi) and binarized, which results in minor artifacts. However, all checkboxes are approximately 80-by-80 images, and the threshold of 2500 foreground pixels separates the checked from unchecked checkboxes well.

Data is analyzable if it is amenable to statistical analysis. The process of data preparation and normalization encompasses

1. 1.

   Identification of captured variables.

2. 2.

   Classification of variables into common classes: quantitative, ordered categorical, unordered categorical.

In addition, to capture tens to hundreds of millions of records present in the dataset we need to design a database schema which will permit efficient statistical analysis and be suitable for machine learning.

It is straightforward to identify the variables, as they closely correspond to the fields present in the forms. This leaves the problem of naming the variables that promotes analysis and is compatible with other available databases. An example of a database table with adopted variable naming conventions is presented in Table 3. The variable names are chosen to work well as “identifiers” in most programming languages. However, SQL allows any naming convention by using double quotes: any string is a valid identifier (table column name) as long as it is quoted with double quotes (").

We represented the time series data by a typical table which stores time as one of the variable. It is noted that handling time in the PDF attachments is far from perfect, because handling time correctly is a difficult problem due to varying standards and daylight saving time.

The data from the “PRE-OPERATIVE MANAGMENT” portion of the DCD Flowsheet form require a second table. Thus, one relatively short form may leads to 2 or more tables in the target database. This table has a large number of mostly categorical variables and is best presented in a format which lists variables vertically rather than horizontally, such as JSON shown in Table 2. Another objective of generating JSON is to confirm that our data can be stored in JSON-oriented databases, such as MongoDB.

It should be noted that the data extracted from the PDF attachements is captured in native MATLAB datastructures, which is the directly produced intermediate format resulting from MATLAB programming. Subsequently, APIs have been constructed, so that we can export the data in various formats. The format of the exported data is compatible with industry-standard RDBMS systems and, at the time of this writing, the relational database consists of 18 tables, quickly growing as new types of medical forms are incorporated. Currently implemented forms are identified by their titles:

1. 1.

   “DCD FLOWSHEET”

2. 2.

   “PRE-OPERATIVE MANAGMENT” (separate table holding a portion of the “DCD Flowsheet”)

3. 3.

   “FLOWSHEET”

4. 4.

   “KIDNEY PERFUSION FLOW SHEET”

5. 5.

   “LIVER DATA”

6. 6.

   “REFERRAL WORKSHEET”

The form with title “FLOWSHEET” is actually a collection of subforms, which we mapped to separate database tables:

1. 1.

   “VITAL SIGNS”

2. 2.

   “VENT SETTINGS”

3. 3.

   “INTAKE”

4. 4.

   “Medications Dosage”

5. 5.

   “OUTPUT”

6. 6.

   “Comments”

The remaining tables consist of book-keeping data, such as patient information, document version information, and timestamps when document was generated.

This data is then exported using MATLAB built-in interfaces to the following data formats

1. 1.

   SQL database (we’re using SQLite via a JDBC driver).

2. 2.

   MongoDB using MATLAB wrapper around ’mongoc’, which is a C-based API for MongoDB, and JSON generated by MATLAB command ’jsonencode’.

3. 3.

   Plain SQL file obtained by exporting the SQLite database to SQL (equivalent to a complete database backup).

4. 4.

   CSV, suitable for a quick export of individual data tables.

Soteria is a HIPAA-compliant HPC system, which holds the OPTN dataset (HIPAA-compliance is mandated). The system features 96 CPU and GPU (currently not used in our project), running a flavor of Linux. The software is build on top of MATLAB and its add-on packages called Toolboxes. Full version of MATLAB with toolboxes is a rich programming environment. When installed, it occpuies 20-30 GB of disk space. We call Java libraries, such as PDFBox and in-house built Java packages, from MATLAB, thanks to a very functional MATLAB-Java interface.

As an example, as 36-page PDF attachment is processed in $4.6$ seconds on a single CPU. This size is representative of the avarage size of an attachment, but they widely vary in size. Thus, the 2022 dataset may take $4\times 10^{5}\times 4.6=1,840,000$ seconds, or $511$ hours. Given that files can be processed in parallel (“trivial parallelism”) the real time of processing can be reduced to as little as $511/96=5.324$ hours. In practice, the time is approximately $1/3$ of that because many attachments can be skipped (e.g., image-based PDFs).

It is clear that software performance is not an issue, as the entire national organ transplant dataset can be processed in $15\times 5=75$ hours, i.e. roughly 3 days. Hence, our system architecture is suited to fully addressing the problem at hand.

The methodology for image-based PDF is different, as it required OCR. OCR, by its nature, does not recover text with absolute accuracy, but our preliminary results indicate that, combined with NLP techniques (e.g., a custom spell-checker) and custom training on the fonts that are actually used in the OPTN forms, we demonstrated near-100% accuracy on selected forms, e.g, “RENAL DATA” which contains detailed kidney anatomy information. In future papers we will cover image-based PDF in more detail.

Large Language Models (LLM) are gaining prominence due to the launch of ChatGPT. The data extracted from PDF contains blocks of free-form text containing valuable information. Incorporation of LLMs into our software is ongoing, and will be present the results in future papers.

In U.S. Donor data has been stored in OPTN’s DonorNet as each donor’s profile as individual PDF attachments since 2008. With 57 OPOs using over 25 different forms to capture kidney anatomy and pathology reporting, Over 1 million PDF attachments for 135,000 deceased donors covering 2008 to 2021 are identified.

The article discusses a methodology for extracting and analyzing data from the PDF attachments, specifically focusing on medical forms like DCD flowsheets. The process involves converting the PDF documents into analyzable data formats, which are then subjected to image processing and regular grammar-based parsing.

This study used data from the Organ Procurement and Transplantation Network (OPTN). The OPTN data system includes data on all donor, wait-listed candidates, and transplant recipients in the US, submitted by the members of the Organ Procurement and Transplantation Network (OPTN). The Health Resources and Services Administration (HRSA), U.S. Department of Health and Human Services provides oversight to the activities of the OPTN contractor.