On the notion of Hallucinations from the lens of Bias and Validity in Synthetic CXR Images

Medical imaging technologies have enabled precise disease diagnosis without invasive tests, yet radiologists lack access to large amounts of training images due to privacy and security concerns [10] [3]. The lack of high-quality annotated medical imagery datasets provides the impetus for generative imaging models to be created that accurately represent medical concepts while providing compositional diversity [4]. The utilization of open-source medical image datasets presents numerous limitations, which include inconsistencies in image quality, gaps in metadata and clinical information, the use of outdated imaging equipment in some dataset acquisitions, the presence of low-quality images, insufficient expert labeling, and often, restrictions on commercial usage imposed by many open-source datasets[18]. Some performance variations could be due to differences in patient demographics across sites, and the study highlights the need for further investigation in this area. For instance, a study found that the AI algorithm’s performance dropped substantially when evaluated on a different site, indicating limitations in generalizability [19]. Deep learning models can create synthetic data to improve diagnostic accuracy and advance medical imaging techniques[12]. The diffusion model has gained popularity as it better estimates data distribution and sample quality than other methods - making it a potential alternative to GANs when creating images [11]. Today, advanced models like Stable Diffusion[15], DALL-E 2, and Midjourney are now capable of producing impressive-quality images from natural language descriptions.[2][14].

Multi-modal models trained on natural image/text pairs often perform poorly when applied to medical applications [4]. Typically, diffusion models follow a two-stage process: (1) a prior generating CLIP image embedding with a text caption; (2) a decoder (U-net) producing an image conditioned upon the image embedding. In this context, diffusion models offer advantages as decoders due to their computational efficiency and ability to generate high-quality samples [13].

While the license agreements from Stable Diffusion limit its use for medical advice or interpretation, researchers at Stanford used the technique to generate radiology images (RoentGen)[4] to assess if this method can solve the current gap of lack of adequate data in medical imaging. With RoentGen, the researchers could generate realistic synthetic CXR images using free-form text prompts, improving diagnostic accuracy. They found that fine-tuning the model on a fixed training set increases classifier performance by 5% when trained on both synthetic and original images and 3% when trained solely on synthetic images. Furthermore, text-encoder representation capabilities for certain diseases like Pneumothorax were enhanced by 25% through fine-tuning.

The RoentGen study investigated using latent diffusion models to generate medical images in thoracic imaging. The researchers list the limitations encountered, such as difficulty measuring clinical accuracy with standard metrics (e.g., usefulness of image), lack of diversity in generated images (only a small sample considered), and need for improved text prompts (e.g., radiology-specific words or verbatim radiology text). The study or the paper did not discuss the known limitations of medical imaging interpretation, including data diversity, bias, validity, and interpretability of the underlying model performance. Our work examines specific considerations to expand the Stanford research question, “Could Stable Diffusion Solve a Gap in Medical Imaging Data?” [8] from the lens of bias and validity of the generated outcomes.

We rigorously validated our dataset and assessed RoentGen’s performance, initializing it with model weights from its creators. Our primary data source was the MIMIC-CXR dataset, comprising CXR images and radiologist textual inputs [1]. An experimental framework was then established, utilizing pre-trained RoentGen model weights, ensuring a robust foundation for image generation. We subsequently generated CXR images based on radiologist textual descriptions from the MIMIC-CXR dataset, with and without protected variables such as Gender and Race as counterfactuals. To gauge image quality and fidelity to medical conditions, we employed a disease classification model trained on multiple- CXR illness categories [5]. The classifier’s performance on original and generated images was compared, providing a comprehensive assessment of the model’s representational capabilities. Additionally, we probed for latent hallucinations by subjecting both original and generated images to a dedicated COVID classifier [9]. As the original images were not expected to show signs of COVID, any discrepancies in the classifier’s results signaled potential features introduced by RoentGen’s hallucinations. We then rigorously compiled and analyzed the results from our experiments.

Interpretation of Generative AI models is challenging and newer methods such as per-word attribution [17] or per-head cross attention [7] have been proposed. It is essential to have an extensive system for comprehending text-to-image attribution to detect any potential flaws or inconsistencies within these models in medical imaging. The harms contributed by the inherent biases in these pre-trained models may have accentuated implications considering the general lack of adequate interpretability of these models. Given the risks noticed from the experimental results, we believe there is a need to have more comprehensive approaches for evaluating the suitability of using Generative models for data augmentation in disease diagnosis.

The specific aberrations arising out of hallucinations from our experimental study were not identified through traditional evaluation metrics for generative images, thereby requiring domain-specific consideration of alternative methods. We choose an out-of-training class validation method to demonstrate the hallucination; however, such an approach may not be suited for many circumstances. We believe these experimental results will trigger further research on establishing validity and reliability metrics for hallucinations in Generative Medical Images. It is also pertinent to consider the need for extended research on approaches to limit latent hallucinations, as some of these hallucinations, if not constrained, can have significant downside impacts on people and the planet.

Our research presents a comprehensive exploration of the drawbacks associated with the utilization of generative AI models in the context of medical imaging. Specifically, our study sheds light on issues related to bias, validity, and the phenomenon of hallucination within the datasets generated by these AI models.

It is important to acknowledge that our study, while serving as a reminder of the longstanding problem of fairness in the realm of medical AI, may inadvertently contribute to the perpetuation of skepticism surrounding the applicability of diffusion models in the medical domain. This skepticism may erode the credibility of diffusion models as a viable and convenient method for enhancing medical image datasets, potentially dissuading other researchers in the field from exploring these avenues.

Nevertheless, in light of the ethical imperatives inherent to the use of AI in medical applications, our research also serves as an illustrative example of the notion that reliance solely on generated datasets to bridge existing gaps may not yet be a sufficiently robust approach.