AI-Driven Healthcare: A Survey on Ensuring Fairness and Mitigating Bias

AI has significantly transformed cardiology by enhancing diagnostic accuracy, personalizing treatment, and improving patient outcomes ^{76, 77}. AI algorithms are adept at analyzing cardiovascular data to facilitate early detection and diagnosis of heart diseases, including arrhythmias, heart failure, and coronary artery disease ^{78, 79}. For instance, machine learning models can interpret echocardiograms with high precision, often surpassing human experts in diagnostic speed and accuracy ⁸⁰. AI also assists in managing cardiovascular risk by integrating diverse patient data to predict individual risk factors more effectively ⁸¹. Furthermore, AI plays a crucial role in developing personalized treatment plans by analyzing patient data and predicting responses to various treatment modalities, optimizing therapeutic decisions ⁸². The integration of AI in clinical practice necessitates rigorous validation and ethical alignment to ensure patient safety and data security ⁸³. The continuous evolution of AI technologies promises further innovations in diagnostic tools and therapeutic strategies, enhancing precision and predictive capabilities in whole person care whereby patients may be prescribed lifestyle changes specific to their context as seamlessly as medications tuned to their genomics ⁸⁴. Moreover, AI can contribute to health equity by making advanced cardiac care accessible in underserved areas through telemedicine platforms and remote monitoring, thereby reducing disparities in cardiovascular health outcomes ⁸⁵.

AI has revolutionized ophthalmology by enhancing diagnostic accuracy, improving patient outcomes, and streamlining clinical workflows. More and more, AI algorithms, especially deep learning models, are being used to look at complicated visual data from imaging methods like fundus photography and optical coherence tomography (OCT) ⁸⁶. These models are very good at diagnosing common eye diseases like diabetic retinopathy, glaucoma, and age-related macular degeneration; they can sometimes match or beat the performance of experts ⁸⁷. Automated analysis using AI not only speeds up the diagnostic process but also reduces human error, facilitating earlier and more precise interventions ⁸⁸. Furthermore, AI’s predictive capabilities are being harnessed to forecast disease progression, which is crucial for conditions like glaucoma, where early detection can prevent severe vision loss ⁶. Using AI tools in clinical practice also includes using AI-driven decision support systems that help plan treatments by guessing how different types of treatments will work ⁸⁹. Despite these advancements, challenges remain, including data privacy concerns, the need for large annotated datasets for training algorithms, and the integration of AI into existing clinical workflows ⁹⁰. However, ongoing research and collaboration between AI technologists and ophthalmic experts are likely to overcome these obstacles, solidifying AI’s role in modern ophthalmology ⁹¹. This further opens the frontier for expansion of access to ophthalmological care in existing deserts through technology-enabled care models and non-specialist operators at the point of service, thereby addressing disparities in eye care availability and quality.

AI is transforming dermatology by enhancing diagnostic accuracy, personalizing treatment plans, and streamlining patient management. AI-powered tools, particularly CNNs, are highly effective in diagnosing skin cancer, matching or surpassing dermatologists in identifying melanomas ^{92, 93}. Moreover, AI applications in dermatology extend beyond cancer detection. They include the assessment and management of chronic conditions such as psoriasis and atopic dermatitis, where AI algorithms help in monitoring disease progression and response to treatment ⁹⁴. AI systems are also utilized in cosmetic dermatology, optimizing treatment recommendations based on individual facial analysis ⁹⁵. Ensuring the efficacy and safety of AI tools requires rigorous validation processes with diverse datasets to mitigate bias and enhance generalizability ¹⁴. Integration into clinical workflows and user training are crucial to maximizing the benefits of AI in dermatological practice ⁹⁶. As AI continues to evolve, continuous collaboration between technologists and clinicians is essential to address ethical considerations and improve patient outcomes in dermatology. Importantly, AI can democratize access to high-quality dermatological care by enabling remote consultations and diagnostics, thus reducing barriers for individuals in remote or underserved regions.

AI has significantly advanced the field of neurology by enhancing diagnostic accuracy, personalizing treatment plans, and facilitating neurological research. AI techniques, particularly machine learning and deep learning, are pivotal in diagnosing neurological disorders such as Alzheimer’s disease and epilepsy by analyzing imaging data with high precision ^{97, 5}. These technologies improve the interpretation of MRI and CT scans, often achieving higher accuracy than traditional methods ⁶. In treatment, AI algorithms personalize therapies based on patient data, optimizing treatment outcomes for diseases like Parkinson’s ⁹⁸. Additionally, AI has revolutionized neuroprosthetics by creating adaptive interfaces, significantly improving the quality of life for patients with motor disabilities ⁹⁹. Predictive analytics also play a crucial role in anticipating disease progression for conditions such as multiple sclerosis ¹⁰⁰. AI’s application extends into research, facilitating the understanding of complex neurological phenomena and the development of innovative treatments ¹⁰¹. Overall, AI’s integration into neurology not only improves clinical practices but also opens new avenues to address rare diseases as well as more finely tune medications to the individual ¹⁰². Furthermore, AI can enhance health equity in neurology by enabling remote diagnosis and monitoring, ensuring that patients in underserved areas receive timely and accurate neurological ¹⁰³.

AI is revolutionizing radiology and cancer treatment by enhancing imaging accuracy, enabling personalized therapy, and improving diagnostic workflows. AI algorithms, particularly deep learning models, have significantly improved the precision of image interpretation in radiology, aiding in the early detection and diagnosis of various cancers, such as breast and lung cancer ¹⁰⁴. These models analyze imaging data much more rapidly than human radiologists, reducing diagnostic times and increasing efficiency ¹⁰⁵. In cancer treatment, AI assists in formulating personalized treatment plans based on patient data and predictive analytics, which can predict treatment outcomes and suggest optimal therapies ¹⁰⁶. AI applications in radiology extend to prognostic evaluations, predicting disease progression and survival rates, refining treatment protocols and developing follow-up strategies  ¹⁰⁷. These advancements are supported by large datasets and strict validation processes, ensuring reliability and clinical applicability ¹⁰⁸.

AI stands to significantly transform emergency medicine and critical care by enhancing patient triage, treatment efficacy, and time to diagnosis. As AI-powered tools like diagnostic algorithms have been incredibly good at the early identification of conditions like sepsis and its progression, there is the potential to improve patient outcomes ^{109, 110}. The prediction of patient outcomes has also positioned AI to facilitate the optimization of critical resource allocation. During the COVID-19 pandemic, AI-based self-triage tools had a critical role to play in predicting cases and hospitalizations at the population level ¹¹¹. AI-powered chatbots and virtual assistants are streamlining patient intake and symptom assessment ^{112, 113}. Furthermore, AI’s role in triaging patients based on urgency has revolutionized care prioritization, improving ED operations and patient flow ^{114, 115}. These advancements underscore AI’s potential to not only enhance emergency medical services but also pave the way for a more efficient, accurate, patient-centered healthcare system that extends care beyond the traditional brick-and-mortar facilities.

Data Bias: Data bias in AI healthcare systems can severely compromise their effectiveness and fairness. For instance, a study by Obermeyer et al. ¹² found that a healthcare management algorithm disproportionately favored white patients over black patients with similar health conditions due to reliance on historical healthcare cost data, which reflected existing socioeconomic disparities. Further studies reinforce the pervasive nature of data bias in healthcare AI. For example, Rajkomar and his team found that the models trained on patient datasets from one geographical region often failed to generalize to populations outside that region, indicating a geographic data bias that limits the practical utility of AI systems in diverse settings ⁴. Additionally, the AI system used for diagnosing diabetic retinopathy performed well in tests but showed decreased accuracy in clinical settings, especially among minority populations not well represented in the training data, underscoring the impact of demographic data biases ¹¹⁶.

Algorithm Bias: Algorithmic bias in AI healthcare occurs when AI systems develop systematic errors that disadvantage certain patient groups. For example, AI models used to diagnose pneumonia displayed biases against specific racial groups, failing to capture important socio-economic and demographic factors despite extensive training on hospital data. This led to inappropriate treatment recommendations ^{4, 13, 117}. Another study in dermatology faced the same issue. The CNNs used for detecting skin cancer were less effective for individuals with darker skin. This discrepancy arose because the training data predominantly consisted of images featuring lighter skin tones, which does not reflect the diversity of global skin types. Consequently, this training limitation can lead to poorer health outcomes for underrepresented groups ¹⁴.

Explicit Bias: Explicit bias in AI systems within healthcare refers to observable prejudices that directly influence these systems’ decision-making processes. These biases are often a result of human prejudices being encoded in to AI through biased data or algorithms, leading to discriminatory practices and unequal treatment of patients ¹¹⁸. One example of explicit bias in healthcare AI is the underdiagnosis of certain demographic groups by AI models. A study investigating the fairness of vision language foundation models in medical imaging found that these AI systems consistently underdiagnosed marginalized groups, such as black female patients, across a range of pathologies ¹¹⁹. This explicit bias in diagnostic AI tools can lead to delayed or inadequate medical care for specific patient subgroups, exacerbating healthcare disparities. Underlying Clinical Decision Support systems, explicit bias may be due to the selection of data and algorithm design, leading to biased outcomes. For example, the decreased likelihood of receiving diagnostic imaging for nonwhite groups compared to their white counterparts when controlling for patient and facility level factors is embedded in clinical data ¹²⁰. Accordingly, models predicting imaging use may reflect the human prejudices around who deserves and receives diagnostics ¹²¹. Moreover, if an AI system is trained predominantly on data from one ethnic group or gender, it may not perform accurately for other ethnicities or gender. The use of biased AI in healthcare can result in systematic discrimination, where certain groups received suboptimal care due to the AI’s prejudiced programming. This can manifest in various ways, such as through biased diagnostic tools, treatment recommendations, or predictive analytics that unfairly disadvantage certain groups.

Implicit Bias: Implicit bias in AI systems in healthcare refers to the unconscious prejudices that can influence decision-making processes without explicit acknowledgment. These biases often reflect societal stereotypes and may not align with the conscious values or beliefs of the developers or users of the AI system. Implicit biases can alter perceptions and judgments, leading to disparities in healthcare delivery and outcomes ¹¹⁹. In healthcare settings, implicit biases can manifest in various ways. For example, a study on obstetric care providers found that implicit biases favored French women over African migrant women, potentially influencing medical decisions and leading to healthcare disparities ¹²². Similarly, healthcare professionals have been shown to exhibit implicit biases based on ethnicity, which could affect the likelihood of patients being given another opportunity to attend an outpatient clinic after missing an appointment ¹²³. Another example is the use of AI in diagnostic tools, where implicit biases can result in differential accuracy rates across racial or ethnic groups. This can lead to underdiagnosis or misdiagnosis for certain populations, contributing to unequal treatment and outcomes ¹²⁴. Implicit biases can also affect the development of clinical decision support systems. This is because the data used to train AI might not fully reflect the diversity of the patient population, which could lead to recommendations and treatment plans that aren’t based on facts ^{125, 126}. Furthermore, implicit biases can impact the patient-provider relationship, where unconscious prejudices may influence the level of care and communication provided to patients of different racial or ethnic backgrounds, potentially affecting their health outcomes ¹²⁷.

Selection Bias: Selection bias occurs when the training data for an AI model is not representative of the broader population or the context in which the model will be applied. This leads to AI systems performing well on training data but poorly in real-world settings, particularly for underrepresented groups ¹²⁸. The development of AI models for disease prediction or diagnosis, primarily trained on data from specific demographic groups like a particular race, gender, or age group, is one example of selection bias in healthcare settings. If an AI model for diagnosing skin cancer is trained primarily on images of lighter-skinned individuals, it may perform poorly when diagnosing conditions on darker skin, leading to higher misdiagnosis rates for individuals with darker skin tones ¹²⁵. AI models that predict patient outcomes or treatment efficacy also exhibit selection bias. If the historical data used to train these models predominantly come from patients who received a certain level of care or who have specific socioeconomic backgrounds, the AI’s predictions may not accurately reflect outcomes for patients outside of these groups ¹²⁸. This can lead to disparities in healthcare recommendations and interventions offered to patients from different socioeconomic or racial backgrounds. Furthermore, selection bias can also manifest in AI-driven tools for patient triage and resource allocation. If the data used to train these models does not adequately account for the diversity of patient presentations and conditions, the AI system may prioritize certain patient groups over others, inadvertently exacerbating existing healthcare disparities.

Biases in AI systems, particularly in healthcare, can have profound and far-reaching consequences. These biases usually stem from the data on which the AI systems are trained, the design of the algorithms themselves, and the contexts in which they are deployed (as discussed in Section 3.1). Here are several potential consequences of biases in AI systems in healthcare:

Misdiagnosis and Inequitable Healthcare Outcomes: AI integration in healthcare brings significant advancements but also risks of misdiagnosis and inequitable outcomes due to biased algorithms. AI systems often rely on non-representative data, leading to biased decision-making. For instance, Obermeyer highlighted that an algorithm used in healthcare disproportionately favored white patients over black patients because it used healthcare costs as a proxy for healthcare needs, indirectly embedding racial biases in its predictions ¹². Moreover, these biases in AI can exacerbate existing healthcare disparities. Rajkomar and others discussed how AI applications, if not carefully designed and monitored, could inherit and amplify socioeconomic and racial disparities. This is particularly problematic in diagnostics, where AI systems are trained predominantly on data from specific demographic groups, potentially leading to poorer diagnostic accuracy for underrepresented groups ⁴. This was evident in a study by Adamson and Smith, which found that dermatology AI systems demonstrated lower accuracy rates in skin lesion diagnosis for dark-skinned individuals compared to those with lighter skin ¹⁴. Such biased AI not only risks misdiagnosis but also contributes to inequitable healthcare outcomes by potentially steering healthcare resources away from those who may need them most. Vayena and his team emphasized the ethical imperative to ensure that AI tools in healthcare are developed with consideration for fairness and equity, advocating for diverse and inclusive data sets and algorithmic transparency to mitigate these biases ¹²⁹.

Loss of trust in Healthcare Systems: Recent studies have echoed the concern that the deployment of biased AI in healthcare could significantly undermine public trust in medical systems. Trust is foundational to the patient-provider relationship and is crucial for the effective delivery of healthcare services ¹³⁰. When AI tools exhibit bias, whether in diagnosis, treatment recommendations, or patient management, they can lead to misdirected care, fostering distrust among patients, particularly in marginalized communities ⁴. Kherbache and his team pointed out that when patients perceive AI-driven processes as opaque or unfair, their trust in the overall healthcare system may decline ¹³¹. Incidents of AI failures that receive public attention can exacerbate this erosion of trust, leading patients to question the reliability and ethics of using AI in medical decision-making. Researchers like Kerasidou argue that trust is not only about the accuracy of AI but also its alignment with ethical principles that govern healthcare, such as beneficence and non-maleficence ¹³². Furthermore, the lack of transparency in how AI models make decisions can be a major barrier to trust. Blease’s study ¹³³ suggests that without clear communication about how AI tools contribute to healthcare decisions, patients may become skeptical of diagnoses and treatments, fearing that their personal healthcare data could be misused or misunderstood. This potential trust deficit could have severe implications, not just for individual health outcomes but also for public health at large, as mistrust in healthcare systems can lead to lower rates of healthcare utilization, vaccine hesitancy, and poor adherence to medical advice ¹³⁴. Veinot highlights the need for healthcare systems to maintain high standards of accountability and transparency as they integrate AI technologies to mitigate such risks ¹³⁵.

Legal and Ethical Implications: The use of biased AI in healthcare not only poses clinical risks but also entails significant legal and ethical implications. Deploying biased AI systems could lead to legal breaches of anti-discrimination laws. In the United States, for example, the Civil Rights Act and the Americans with Disabilities Act set legal standards that could be violated by biased AI algorithms, which fail to provide equitable care across different patient demographics ¹³⁶. Ethically, biased AI conflicts with the fundamental medical ethics principles of justice and non-maleficence, demanding fairness and avoidance of harm, respectively ¹³⁷. Moreover, biased algorithms in healthcare could potentially expose medical practices and institutions to litigation related to malpractice or negligence, especially if these algorithms contribute to substandard care outcomes ¹³⁸. For instance, if an AI system were to consistently provide inferior diagnostic support for certain racial groups, this could be viewed as a form of systemic negligence or malpractice ¹²⁹. Additionally, the ethical implications extend to the breach of patient trust and the compromise of patient autonomy. Ethical medical practice relies heavily on the principles of informed consent and respect for patients’ autonomy—principles that are challenged by opaque AI systems that do not make their decision-making processes or inherent biases clear to patients or practitioners ¹³³. These potential legal and ethical failures underscore the necessity for rigorous oversight and transparent development processes for AI in healthcare, aiming to ensure that these technologies adhere to both existing legal frameworks and ethical standards of practice.

Resource Misallocation: The deployment of biased AI in healthcare settings can lead to resource misallocation, a critical issue that impacts both the efficiency and fairness of medical services. Biased algorithms may misdirect resources by prioritizing certain groups over others based on flawed data inputs or biased training procedures. For example, a study by Obermeyer demonstrated how an algorithm used for managing healthcare resources inadvertently favored healthier white patients over sicker black patients due to biased data inputs that did not accurately reflect patient needs ¹². This misallocation can exacerbate existing healthcare disparities by diverting necessary medical attention and resources away from those who are most in need. As Chen pointed out, such disparities are not just a matter of clinical outcomes but are deeply tied to social and economic inequalities that AI tools can inadvertently perpetuate ¹³. Furthermore, biased AI can influence the allocation of resources within healthcare facilities, potentially resulting in inefficiencies that strain healthcare systems. This includes misallocating medical staff, diagnostic tools, and hospital beds, which can degrade the quality of care delivered while increasing wait times and healthcare costs ⁴. Resource misallocation also raises ethical questions about fairness and equity in healthcare provisioning. It challenges the ethical principle of justice, which demands that healthcare resources be distributed based on need rather than biased algorithms ¹³⁷. This ethical breach can lead to further mistrust and reluctance among underserved populations to engage with healthcare systems, perpetuating a cycle of disadvantage.

Stifling Innovation: The presence of bias in AI systems used in healthcare not only affects the accuracy and fairness of medical services but can also stifle innovation. When AI models are made using biased data, they might not accurately reflect the different needs of the population. This could make these innovations less useful and applicable to a wider range of demographic groups. Liang talks about how biased training datasets in AI development make it harder for these systems to work with different types of data. This could stymie innovation by preventing it from solving larger or more complex health problems that affect many people. Moreover, reliance on biased AI can deter investment in developing new technologies that are inclusive and equitable. Potential investors may be cautious about funding projects that might not meet regulatory standards for fairness or could lead to public backlash — a concern highlighted by Corbett-Davies and Goel, who note the legal and social implications of deploying biased AI ¹³⁹. In addition, the perpetuation of bias in AI could solidify existing disparities in healthcare innovation. As Benjamin notes, if innovation is directed predominantly at solving problems specific to well-represented groups, less common but equally pressing issues in underrepresented groups are neglected, thereby limiting the scope and impact of technological advancements in healthcare ¹⁴⁰. The studies of Rajkomar ⁴, Hardt ¹⁴¹, and Howell ¹⁴² made things even more complicated by pointing out that biased algorithms can make it harder to find new treatments that work for everyone. This feeds a cycle of innovation that helps groups that are already overrepresented in data sets more than others. This restricted focus on innovation not only affects the equity of healthcare delivery but also limits the development of truly innovative, comprehensive healthcare solutions. These factors combined suggest that biased AI not only hinders the progression of medical technologies but also potentially locks the healthcare sector into a cycle of uneven innovation where only the needs of the majority are systematically addressed.

In healthcare, detecting bias in AI systems is crucial, as it can result in disparities in healthcare delivery and outcomes. This section provides an overview of bias detection strategies in healthcare AI system, including: statistical analysis, auditing tools, and feedback from end-users.

Mitigating biases in AI systems within healthcare is crucial for ensuring equitable treatment across all patient demographics. This section outlines key strategies to address and reduce biases, including the use of diverse and representative data and fairness-aware approaches, thereby enhancing the fairness and accuracy of AI models.

The integration of AI in healthcare presents significant ethical considerations that necessitate careful scrutiny. One primary concern is privacy and data protection, as AI systems typically require large datasets, including sensitive patient information, to train and operate effectively. Ensuring the confidentiality and security of this data against breaches is paramount ¹⁸⁵. Additionally, the issue of bias in AI algorithms poses a serious ethical challenge. AI systems can perpetuate and even exacerbate existing biases if they are trained on non-representative data, leading to unequal treatment outcomes among different demographic groups ⁴. There is also the ethical issue of transparency and explainability. Practitioners and patients may find it challenging to comprehend the reasoning behind AI decision-making processes due to their opaque nature. This lack of transparency can undermine trust in AI systems and complicate informed consent processes ¹⁸⁶. Furthermore, the deployment of AI in healthcare must take into account the potential displacement of healthcare professionals, raising concerns about job security and the loss of valuable human expertise and empathy in patient care ¹⁸⁷. Lastly, accountability for AI-driven decisions in healthcare is a critical ethical issue. It is critical to define who is responsible—the developers, the users, or the AI itself—when an AI system’s decision results in patient harm ¹⁸⁸.

The deployment of AI in healthcare brings forth numerous legal considerations that are crucial for ensuring compliance and safeguarding patient rights. Intellectual property rights stand out prominently, as there is ongoing debate over whether AI-generated medical inventions can be patented and who owns the rights to AI-generated data and algorithms ¹³⁸. Furthermore, liability issues arise when AI systems make erroneous decisions that could harm patients. Determining whether the healthcare provider, the software developer, or another party is liable requires intricate legal analysis and possibly new legal frameworks ¹⁸⁹. Compliance with medical standards and regulatory requirements is another critical legal area. AI applications in healthcare must adhere to existing healthcare regulations, such as the Health Insurance Portability and Accountability Act (HIPAA) in the U.S., which governs the privacy and security of patient data, and the General Data Protection Regulation (GDPR) in the EU, which sets stringent guidelines for data protection and privacy ¹⁹⁰. As AI technologies evolve, there may also be a need for specific regulations that address AI’s unique aspects in healthcare to ensure that these innovations are safely and effectively integrated into medical practice ¹⁹¹. Lastly, issues of informed consent are magnified with AI, as patients must understand the role of AI in their care, including how their data will be used and the implications of AI-driven decisions ¹⁸⁸.

The lack of diversity in training datasets is a significant research gap that impacts AI fairness in healthcare. Models trained on non-representative data can perpetuate or even exacerbate existing biases against underrepresented groups ⁴. To address this, techniques for synthetically augmenting underrepresented classes in datasets or incentivizing the collection of more comprehensive and inclusive data must be developed ¹¹⁷. Additionally, implementing algorithmic fairness approaches during model training, such as fairness constraints or adversarial debiasing, can help mitigate biases introduced by skewed datasets ¹⁹². Future directions also include promoting policy changes that enforce transparency and fairness in data collection and algorithmic processing in healthcare applications ¹³⁷.

The longitudinal effects of AI systems on healthcare fairness remain underexplored, with existing studies often focusing on short-term outcomes and immediate biases in algorithmic decision-making ¹⁹³. Understanding how these technologies affect health disparities over time is crucial, as initial biases can amplify if not properly addressed ⁴. Future research should involve continuous monitoring of AI systems post-deployment to assess their impact on various demographic groups across different time intervals ¹⁹⁴. This approach will help identify evolving biases and enable timely modifications to algorithms, thereby ensuring more equitable healthcare outcomes ¹². Additionally, incorporating feedback loops that allow healthcare providers to report disparities can further refine AI applications in real-world settings.

The integration of AI in healthcare requires interdisciplinary approaches that combine insights from medical ethics, data science, social sciences, and clinical practice to comprehensively address fairness ¹⁹⁵. However, current AI applications often lack this multidimensional perspective, resulting in solutions that are technically adequate but socially or ethically inappropriate ¹³⁷. Future research should focus on frameworks that incorporate not only technical accuracy but also ethical, legal, and social implications from the outset, ensuring that AI systems are developed with a holistic view of fairness ¹³⁵. These collaborative frameworks can help prevent the exacerbation of existing disparities and promote a more equitable distribution of healthcare resources ¹³.

Transparency in AI algorithms is a significant research gap that limits broader adoption and trust in AI-driven healthcare systems. The inherent “black box” nature of many advanced machine learning models, such as deep neural networks, makes their decision-making processes difficult to understand, leading to issues in fairness and accountability ¹⁹⁶. To address this, future research should focus on developing “glass-box” approaches that are inherently interpretable, such as decision trees or linear models, fostering greater trust among healthcare providers and patients ¹⁹⁷. Additionally, implementing model documentation standards like Model Cards can provide essential transparency by detailing the capabilities and limitations of AI models to all stakeholders ¹⁹⁸.

The impact of AI on healthcare access represents a critical research gap, as disparities in AI-driven healthcare solutions can exacerbate existing inequalities. Studies suggest that geospatial AI models, when applied to geographic information system (GIS), can uncover regions with inadequate healthcare services, thus guiding interventions to those most in need ¹³⁵. However, predictive analytics must be carefully designed to ensure they do not perpetuate biases present in underlying data ¹². Integrating feedback mechanisms from healthcare providers into AI systems offers a pathway to refine these models, ensuring they adapt to serve diverse populations effectively. The future direction in bridging this gap involves developing AI technologies that are not only predictive but also prescriptive, providing actionable insights to improve universal healthcare accessibility ⁴.

The interpretability of AI decisions is a significant research gap in ensuring fairness in healthcare applications. Complex AI models, such as deep neural networks, often act as “black boxes”,making it difficult for healthcare providers to understand and trust their outputs ¹⁹⁷. Enhanced interpretability is critical for trust and verifying the fairness of AI outcomes across diverse patient groups ¹⁹⁶. Future research should focus on adopting model-agnostic explanation frameworks like LIME ¹⁹⁶ or SHAP ¹⁹⁹, which provide insights into the decision-making processes of any AI model, regardless of its underlying architecture. Developing these tools will enable clinicians to better understand and oversee AI-driven decisions, potentially leading to more equitable healthcare outcomes.