On the Gap between Epidemiological Surveillance and Preparedness

A cognitive DSS is a semi-automated system, which deploys CI to process the data sources and to assess risks or biases; this assessment is submitted to a human operator for the final decision.

The risk, reliability, trust and bias measures are used in ES in simple forms such as ‘high-risk group’, ‘risk factor’, and ‘systematic difference in the enrollment of participants’ [24]. However, experts expect from the cognitive DSS more detailed assessments of epidemic scenarios because the corpus of risk terms and definitions is limited. For example, syndrome surveillance consists of real-time indicators for disease that allow for early detection. The DSS must support an expert with answers to the following questions: How risky a given state of ES with respect to collapse of health care resources? Can experts rely on the collected data? What kind of biases can be expected in data collection, algorithmic processing, and the decision making?

For example, in automated decision making, and in our study, the Risk is defined as a function $F$ of cost (or consequences) of a circumstance or event and its occurrence probability:

$$\texttt{Risk}=F(\texttt{Cost},\ \texttt{Probability})$$

For example, in the context of detecting or testing for an infectious disease, the bias is related to the sampling approaches (e.g. tests are performed on a proportion of cases only), sampling methodology (systemic or random), and chosen testing procedures or devices [25].

While all these biases are different, they are probabilistic in nature because the evidence and information gathered to make a decision is always incomplete, often inconclusive, frequently ambiguous, commonly dissonant, conflicting, and has various degrees of believability. Identifying and mitigating bias is essential for assessing decision risks and AI biases [16, 26, 27].

For the cognitive DSS to operate effectively on the human’s behalf, the system may need to use confidential or sensitive information of the users such as personal contact information [37]. The users and the operators must be confident that the cognitive DSS will do what they ask, and only what they ask. Human acceptance of the cognitive DSS technology is determined by the combination of the bias factors and risk factors [22, 23]. The contributing factors include belief, confidence, experience, certainty, reliability, availability, competence, credibility, completeness, and cooperation [38, 39]. In our approach, the causal inference platform calculates various uncertainty measures [7] in risk and bias assessment scenarios.

Risk in decision making process manifests itself in various ways, such as the reliability of information sources (e.g. infection survey data) and the credibility of the information (e.g. testing procedure accuracy):

$\underbrace{\texttt{Data~{}Reliability}}_{Quality}~{}{\Leftrightarrow}\left\{\begin{array}[]{c}\texttt{Risk}\\ \texttt{Trust}\\ \texttt{Bias}\\ \end{array}\right\}{\Leftrightarrow}~{}\underbrace{\texttt{Test~{}Credibility}}_{Reputation}$

This relationship can be represented as follows:

• $(a)$

  Source reliability as the quality of being reliable, or trustworthy, is related to 1) risk as a function of potential adverse impact and the likelihood of occurrence, 2) the confidence in quality, and 3) bias as systematic over- or under-assessment of the parameter of interest.

• $(b)$

  Information credibility as the reputation impacting one’s ability to be believed. In ES, in particular, it can be associated with the credibility of infection testing technology.

These metrics of uncertainty closely resemble the ones known as the Admiralty Code defined in the NATO Standardization Agreements such as STANAG 2022 and STANAG 2511 [35, 33, 34] (Fig. 3). NATO uses the Admiralty Code to resolve conflicting scenarios in human-human, human-machine, and machine-machine interactions. The reliability of the DSS can be increased by using more reliable sources and creditable information, or it can be diminished due to lowered reliability of the source or/and credibility of the information. For example, scenario $C4$ in Figure 3 is composed as the source reliability C$\equiv$ <Fairly Reliable> and information credibility 4$\equiv$ <Doubtful true>. In this context, risk can be expressed in terms of the fair reliability of data sources while doubtful credibility of information.

In this study, we argue that similar standards shall be developed in the ES practice. Definition of measure of uncertainty in the ES will allow to formalize the reasoning upon uncertainty, and define the variables to be included in causal networks, and the associated probabilities. In addition, various scenarios developing in epidemic situations, can be characterized by various levels of uncertainty.

Fig. 4 explains the DSS mechanism for assessing different scenarios that are called the system states. For example, consider the states $\{S_{1},S_{2},\ldots,S_{12}\}$ of the ES. The DSS analysis accordingly to the Admiralty Code (Figure 3) results in the following decision-making landscape:

$-$ States $S_{1}$ and $S_{9}$; $S_{2}$ and $S_{7}$; $S_{3},S_{4}$, and $S_{5}$ can be used for decision-making; $-$ States $S_{10}$; $S_{6}$ and $S_{11}$; and $S_{12}$ results in high-risk decisions.

A cognitive DSS for ES support is a complex dynamic system with the following elements of a cognitive system [29]:

1. Perception-action cycle that enables information gain [30];

2. Memory distributed across the entire system (personal data are collected in the physical and virtual world);

3. Attention is driven by memory to prioritize the allocation of available resources; and

4. Intelligence is driven by perception, memories, and attention; its function is to enable the control and decision-making mechanism to help identify intelligent choices. These cognitive elements are distributed throughout the system in a multi-state perception-action cycle [13, 8].

In most generic sense, the perception-action cycle of a cognitive dynamic system consists of an actuator, an environment, and a perceptor that are embodied in a feedback loop [29]:

• $-$

  The actuator is represented by expert team and their DSS; they make decisions and actions using available perceived information.

• $-$

  The environment is represented by an epidemiological evidence (real or modeled).

The DSS design process including performance evaluation is shown in

Figure 5. The important parts of this process include system “precaution” measurement of the models used for measurement of uncertainty, as well as the DSS precaution measurement in terms of risks and biases with respect to the recommended decisions.

In our work, given a certain causal network platform, the reasoning operations are defined as follows:

1) State assessment, such as the risk of the data source being unreliable. In modeling, risk is represented by a corresponding probability distribution function.

2) Causal analysis is based on the “cause-effect” paradigm. In particular, Granger causality analysis is an advanced tool for this purpose [47].

3) Risk reliability, trust and bias learning. In [48], an approach to learning the trust model has been proposed.

4) Risk, trust and bias propagation. The risk propagation problem was studied, in particular, as a multi-echelon supply chain problem in [52]. In [49], the trustworthiness is propagated through a three-layer model consisting of a source layer, evidence layer, and claim layer. The quality of evidence was used to compute the trustworthiness of sources. Trust can be propagated through a subjective network [36].

5) Reasoning is the ability to form an intelligent conclusion or judgment using the evidence. Causal reasoning is a judgment under uncertainty based on a causal network [5].

6) Risk, reliability, trust and bias adjustment (or calibration) aims to improve the confidence of the risk assessment. For example, negative testing in a symptomatic patient may result in a high risk indicator, but can be later adjusted using additional testing.

7) Risk, reliability, trust and bias mitigation. In most scenarios, the result of intelligent risk processing is a reduction in risk. Risk is lowered via mitigation measures and periodical re-assessment in ongoing screening processes.

8) Risk prediction. In complex systems, meta-recognition, meta-learning, and meta-analysis can be used to predict the overall success (correct assessment of the risk) or failure (incorrect one). The most valuable information for such risk assessment is in the “tails” of the probabilistic distributions [50, 51].

A causal network is a directed acyclic graph where each node denotes a unique random variable. A directed edge from node $n_{1}$ to node $n_{2}$ indicates that the value that is attained by $n_{1}$ has a direct causal influence on the value that is attained by $n_{2}$. Uncertainty inference requires data structures that will be referred to as Conditional Uncertainty Tables (CUTs). A CUT is assigned to each node in the causal network. Given a node $n$, the CUT assigned to $n$ is a table that is indexed by all possible value assignments to the parents of $n$. Each entry of the table is a conditional “uncertainty model” that varies according to the choice of the uncertainty metric.

Analysis of a causal network is out of the scope of this paper. However, we introduce in this paper the systematic criteria for choosing the appropriate computational tools. In addition, some details are clarified in our experimental study.

Thee are multiple approaches to perform case-and-effect analysis under uncertainty using the causal network structure. Depending on the type of uncertainty measure, they causal networks can be divided to [45]:

Bayesian [5], Interval [41]; Imprecise [40]; Credal [42]; Dempster-Shafer [43]; Fuzzy [44]; and Subjective [36].

The type of a causal network shall be chosen given the DSS model and a specific scenario. The choice depends on the CUT as a carrier of primary knowledge and as appropriate to the scenario. There were several attempts to provide researchers with the “Guidelines” for choosing the best causal network platform based on the CUT. Comparison of causal computational platforms for modeling various systems is a useful strategy, such as Dempster-Shafer vs. credal networks [46], and Bayesian vs. interval vs. Dempster-Shafer vs. fuzzy networks [7, 8].

Motivation of choosing Bayesian causal networks is driven by the following:

• $-$

  a DSS can be described in causal form using cause-and-effect analysis.

• $-$

  Bayesian (probabilistic) interpretation of uncertainty provides an acceptable reliability for decision-making.

The Bayesian decision-making is based on evaluation of a prior probability given a posterior probability and likelihood (event happening given some history of previous events)

$$\overbrace{P(\texttt{Hypothesis|Data})}^{Prior}=$$ $$\overbrace{P(\texttt{Data|Hypothesis})}^{Likelihood}\times\overbrace{P(\texttt{Hypothesis})}^{Posterior}$$

Let the nodes of a graph represent random variables $X=\{x_{1},\ldots,x_{m}\}$ and links between the nodes represent direct causal dependencies. A Bayesian causal network is based on a factored representation of joint probability distributions in the form

$\displaystyle P(X)=\overbrace{\prod_{i=1}^{m}P(x_{i}|\overbrace{\texttt{Par}(x_{i})}^{Nodes})}^{Factorization}$

where $\texttt{Par}(x_{i})$ denotes a set of parent nodes of the random variable $x_{i}$. The nodes outside $\texttt{Par}(x_{i})$ are conditionally independent of $x_{i}$. Hence, the Bayesian network has a structural part reflecting causal relationships, and a probability part reflecting the strengths of the relationships. Factoring techniques have been applied to the construction of the Bayesian network.

The posterior probability of $A$ is called the belief for $A$, $\texttt{Bel}(A)$. The probability $P(a|b)$ is called the likelihood of $b$ given $a$ and is denoted $L(b|a)$.

The presence of epidemic/pandemic uncertainties is equally unavoidable; hence, experts require a CI approach that accounts for uncertainty. Probabilistic reasoning on causal (Bayesian) networks enables knowledge inference based on priors and evidence has been applied to diagnostics for precision medicine [28].

Most recently, COVID-19 test-specific risk analysis was performed in [57]: the Bayesian inference was applied to learn the proportion of population with or without symptoms from observations of those tested along with observations about testing accuracy.

An example of a simplified causal network that can serve as a framework for risk inference in the context of preparedness is shown in Figure 6. The factors affecting the risk can be evaluated and inferred for various considered scenario. The derivatives such as Preparedness Index can be derived in a similar fashion.

Note that this is only a fragment of a causal network that may be used in the DSS to assess risks related to outbreak itself. A DSS to be used in Epidemic Preparedness system shall include a system of DSSs. Each expert needs a specific-purpose DSS related to their respected field of expertise. Given data from the epidemiological model, the output of each DSS is a result of a dynamic evidential reasoning. The general principles of building such DSS, as outlined in this paper, shall be applied to build the aforementioned hierarchy of DSSs for better managing future epidemics and pandemics.