Generalized Constraints as A New Mathematical Problem in Artificial Intelligence: A Review and Perspective

When using Bayesian tools, noninformative prior [44] and maximum entropy prior [45] are suggested for realizing a higher degree of objectivity in reasoning.

When addressing an inverse problem, regularization is an important set of GPs to solve such ill-posed problem [46, 47, 48]. When $L_{1}$ and $L_{2}$ penalties are used for the Lasso and ridge methods, they are corresponding to different GPs, sparseness [49] and smoothness [50], respectively. For tensor (or matrix) approximations, a regularization penalty can be a low-rank [51], or further GPs, such as, structured low-rank with nonnegative elements [52]. A semantic based regularization is expressed by a set of first-order logic (FOL) clauses [53].

Knowledge or fuzzy rules [54, 55] provide a structured representation in embedding human semantic knowledge into the machines. Zadeh proposed a GTU framework [7, 8] in order to enlarge GCs by covering other theoretical tools like rough sets [56], belief functions [57], etc.

Abu-Mostafa [58] was one of pioneers of proposing GCs in AI, but used notions of “intelligent hints” and “common sense hints” [59]. He summarized five types of hints, namely,

1. 1.

   invariance,

2. 2.

   monotonicity,

3. 3.

   approximation,

4. 4.

   binary,

5. 5.

   examples,

in learning unknown functions, and developed a canonical representation of the hints [58]. He pointed out that “Providing the machine with hints can make learning faster and easier” [59]. The important implications of his study are:

• •

  In AI modeling, one needs to apply various types of hints, or GCs, as much as possible “from simple observations to sophisticated knowledge” [59].

• •

  A mathematical and canonical representation of hints is necessary so that a learning algorithm is able to deal with hints numerically in a unified form.

In [60], Jain presented several GPs in clustering studies, such as, criteria based on the Occam’s razor principle for determining the number of clusters (say, AIC, MDL, MML, etc), sample priors (say, must-link, cannot-link, seeding, etc). In unsupervised ranking of web pages in search engine studies, Google PageRank applied a semantic GP: “More important websites are likely to receive more links from other websites” [61], based on the link-structure data. When without such data in unsupervised ranking of multi-attribute objects, Li et al. [62] suggested five GCs, in the notion of “meta rules”, that should be satisfied for the ranking function, that is,

1. 1.

   scale and translation invariance,

2. 2.

   strict monotonicity,

3. 3.

   compatibility of linearity and nonlinearity,

4. 4.

   smoothness,

5. 5.

   explicitness of parameter size.

The important implication of the studies in unsupervised ranking is gained below.

• •

  Meta rules, or GPs, are important for assessing learning tools in terms of “high-level knowledge” [62], and become critically necessary when no ground truth (say, labels) exists.

In [63], Lauer and Bloch presented a comprehensive review of incorporating priors into SVM machines. They considered two main groups of priors, namely, class invariance group and knowledge on data group. In [64], Krupka and Tishby took the notion of meta features, and listed several of them in association with the specific tasks. For example, the suggested GPs in the handwritten recognition are

1. 1.

   position,

2. 2.

   orientation,

3. 3.

   geometric shape descriptors;

and in text classification are

1. 1.

   stem,

2. 2.

   semantic meaning,

3. 3.

   synonym,

4. 4.

   co-occurrence,

5. 5.

   part of speech,

6. 6.

   is stop word.

Considering a regression problem, Hu et al. [2] considered GCs in the notion of PKR. They listed ten types of PKR about nonlinear functions in a regression problem, that is,

1. 1.

   constants or coefficients,

2. 2.

   components,

3. 3.

   superposition,

4. 4.

   multiplication,

5. 5.

   derivatives and integrals,

6. 6.

   nonlinear properties,

7. 7.

   functional form,

8. 8.

   boundary conditions,

9. 9.

   equality conditions,

10. 10.

    inequality conditions.

For example, the first type of PKR, a physical constant, was studied from the Mackey-Glass dynamic problem in a difference form:

Within the observation data of the dynamics, one is able to have the prior for a time delay $\tau$, to be a positive, yet unknown, integer in the problem. They showed solutions to gain the estimations of physical constants $\tau$ and $b$ using GCNN model based on radius basis function (RBF) networks. In [42], Qu and Hu further studied “linear priors (LPs)” within GCs, which was defined as “a class of prior information that exhibits a linear relation to the attributes of interests, such as variables, free parameters, or their functions of the models”. The total 25 types of LPs were listed in regressions.

Virtual samples are one of the important GPs given in either an explicit or implicit form in modeling. A few of approaches are listed below.

1. 1.

   Virtual views [21]: generating virtual views from a real view image,

2. 2.

   Adding noise [65, 63]: adding noise types and levels into the input, output, and connection weights of ANNs for robust predictions,

3. 3.

   Graphical model [66, 67]: a framework of generating virtual data with probability,

4. 4.

   SMOTE [68]: an approach of generating samples for the minority class in imbalanced classification,

5. 5.

   Adversarial examples [69, 70]: generating adversarial examples for better representation learning in deep neural networks.

This is a basic mode of incorporating GCs into a model mostly from its data used. In apart from the methods in the virtual samples discussed previously, the other methods appeared, such as,

1. 1.

   Data with labels [3] or ranking list [72],

2. 2.

   Seeding data in learning [73],

3. 3.

   Weight prior [74],

4. 4.

   Cost matrix in imbalance learning [75],

5. 5.

   Universum data [76],

6. 6.

   Spatial context in image patches [77].

This is a basic mode of incorporating GCs into a model mostly from its algorithm used. The methods mentioned previously, say, in objective prior or regularization, fall within this mode. The other methods can be:

1. 1.

   Objective functions and constraints [65],

2. 2.

   Activation functions or kernels [78, 79],

3. 3.

   Weight initialization [80, 81],

4. 4.

   Searching steps [82],

5. 5.

   Meta learning [83],

6. 6.

   Transfer learning [84],

7. 7.

   Curriculum learning and self-paced learning [85, 86],

8. 8.

   Dropout [87].

This is a basic mode of incorporating GCs into a model mostly from its structure used, such as

1. 1.

   Decision tree [88],

2. 2.

   Neocognitron [89],

3. 3.

   Convolutional neural network [90],

4. 4.

   Fuzzy system [91],

5. 5.

   Long short-term memory [92],

6. 6.

   Bayesian networks [93],

7. 7.

   Markov networks [94],

8. 8.

   Knowledge graph [95],

This is a mixed mode by including at lest two basic modes for incorporating GCs into a model, such as

1. 1.

   First-principle + ANN models [9],

2. 2.

   Neuro-fuzzy models [96],

3. 3.

   Connectionist-symbolic models [97, 98],

4. 4.

   Graphical models [66],

5. 5.

   SMOTE + C4.5 [68],

6. 6.

   Markov logic networks [99],

7. 7.

   GCNNs [2],

8. 8.

   Generative adversarial nets [69],

9. 9.

   Graph Convolutional Network [100].

Note that the classification above may be roughly correct to some approaches. For example, we put Bayesian networks within a structural mode to stress on its structural information, but put graphical models within a hybrid mode to stress on its structural information and generation of virtual data.

For coupling forms, we adopt the notion of “Generalized Constraint (GC)” model [11] for explanations. Fig. 7 depicts a GC model, which basically consists of two modules, namely, knowledge-driven (KD) submodel and data-driven (DD) submodel. For simplifying the discussion, a time-invariant model is considered. A two-way coupling connection is applied between two submodels. For stressing on the modeling paradigm, a GC model is considered within the KDDM approach [101]. The general description of a GC model is given in a form of [11]:

where $\mathbf{x}\in\mathcal{R}^{n}$ and $\mathbf{y}\in\mathcal{R}^{m}$ are the input and output vectors, $\mathbf{f}$ is a function for a complete model relation between $\mathbf{x}$ and $\mathbf{y}$, $\mathbf{f_{k}}$ and $\mathbf{f_{d}}$ are the functions associated to the KD and DD submodels, respectively. $\mathbf{\theta}\in\mathcal{R}^{(p+q)}$ is the parameter vector of the function $\mathbf{f}$, $\mathbf{\theta_{k}}\in\mathcal{R}^{p}$ and $\mathbf{\theta_{d}}\in\mathcal{R}^{q}$ are the parameter vectors associated to the functions $\mathbf{f_{k}}$ and $\mathbf{f_{d}}$, respectively. The symbol “$\oplus$” represents a coupling operation between the two submodels.

In fact, we can view the given GCs to be a KD submodel. In 1992, Psichogios and Ungar [9] proposed an idea of using the first principle or empirical function to be KD submodels. When simulating a bioreaction process, they solved partial differential equations (PDEs) by a conventional approach except that a vector of physical parameters was estimated by a neural network submodel. Their HNN model applied the coupling between the two submodels in a composition form:

where the prior was applied implicitly in which the physical parameter vector $\mathbf{\theta}$ was a nonlinear function to the input variables, rather than constant. We can call this form to be Composition I (with parameter for learning), which can include a whole set or a part set of parameters for learning. This study is very enlightening to show an important direction to advance the modeling approach by the following implementations:

• •

  Any theoretical or empirical model can be used as KD submodel so that the whole model keeps a physical explanation to the system or process investigated.

• •

  The DD submodel, using either ANNs or other nonlinear tools, is integrated so that some unknown relations, nonlinearity, or parameters of the system investigated can be learnt.

From the implementations above we can see that coupling form seems to be an extra challenge in the design of HNN or GC models. In 1994, Thompson and Kramer [10] presented an extensive discussions about synthesizing HNN models for various types of prior knowledge. After combining a parametric KD submodel and a nonparametric DD submodel, they called the whole mode to be a semiparametric model. They considered two structures, parallel and serial, in coupling two submodels. Based on the two structures, Hu et al. [2] showed the four coupling forms in Fig. 8, and their mathematical expressions:

The four forms are simple and common in applications. In a parallel structure with a superposition form (Fig. 8(a)), when a KD submodel serves as a core element to simulate the main trends about the process investigated, a DD submodel works as an error compensator for uncertainties from the residual between the KD submodel output and the target output. One good example is a design of a nonlinear Kalman filter in [102], where they applied a linear Kalman filter as a KD submodel and a neural network as a DD submodel to compensate for nonlinear deviations of the state variables. A multiplication form (Fig. 8(b)) was shown on a “Sinc” function in [2]:

where the upper part was known, and the lower part was approximated by a RBF submodel. Hu et al. [2] used this example to show a possible problem introduced by a coupling which is discussed in the next section.

In a serial structure, another two composition forms are shown in eq. (8). The form of Composition II (Fig. 8(c)) corresponds to the inner function to be known as a KD submodel, such as a wavelet preprocessor before a neural network submodel in EEG analysis [103]. The idea of Composition III appeared in [104] to force the output to be consistent with a distal teacher, like a KD submodel $\mathbf{f_{k}}(\mathbf{z},\mathbf{\theta_{k}})$ in Fig. 8(d). Coupling forms can go quite complicated if considering more other structures, such as a cyclic loop in a dynamic process [105], or using a combination of various forms [106, 107].

GC can be given in any representation forms shown in Table I. Actually, GCs in modeling are initially described by natural language. Therefore, in general, a procedure will be involved as defined below:

If GCs are given in a form of mathematical representations, the procedure above will be passed in modeling. However, when GCs are known only in a form of natural language descriptions, this procedure must be processed, like the given example in Table I. The most difficulty in constraint mathematization is due to the problem called semantic gap. When various definitions about it exist from different contexts, such as from retrieval applications in [108], we adopt the general definition:

In [4], Hu suggested consider the semantic gap further by distinguishing two ways of transformations in modeling. A direct way is to transform natural language descriptions into mathematical representations. An inverse way is opposite to the direct one. Hence, the problem of semantic gap can be further extended by the two mathematical problems, namely, ill-defined and ill-posed problems. In [110], Lynch et al. discussed various definitions of an ill-defined problem from the different researchers and proposed their definition below:

Hadamard [111] was the first to define a mathematical term well-posed problems in mathematical modeling. Three criteria are given about well-posed definition to reflect three mathematical properties, namely, existence, uniqueness and continuity, respectively. Based on that, Poggio et al. proposed a simply definition of ill-posed problems below:

Any violation of one of the three properties will result in an ill-posed problem. Constraint mathematization works like a direct way of semantic gap, in which GCs may be ill defined. However, object recognition or constraint learning is an inverse way in which one generally encounters the problems with both ill-defined and ill-posed characterizations.

In [53, 112], good examples were shown in constraint mathematization of first-order logic GCs for kernel machines and DNNs, respectively. However, it may be not easy to conduct constraint mathematization directly. For example, affective computing [113] will face more serious problems of semantic gap and ill-defined problems. The main difficulty sources mostly come from ambiguity and subjectivity of the linguistic representations about emotional or mental entities in modeling. In a study of emotion/mood recognition of music [114], the linguistic terms were used, such as “valence” and “arousal” in Thayer’s two-dimensional emotion model [115], and emotion states like “happy”, “sad”, “calm”, etc. Many low-level and high-level features were used for establishing the relationship between music and emotion states. This study shows that we may need a model for constraint mathematization.

If a GC model (Fig. 7) is used, a new procedure will appear in modeling as:

The subject of coupling form selection has not received much attention in ML or AI studies. Sometimes, the selection is determined by the problem given, like the multiplication form in approximation of “Sinc” function in eq. (9), where its upper part is known. In a general case, for the same GC or KD submodel, there may exist several, yet different, coupling forms to combine with a DD submodel. One example is about monotonicity of the prediction function with respect to some of the inputs. In [116], Gupta et al. presented 20 methods, including their own method. They divided those methods within four categories: A. constrain monotonic functions, B. post-process after training, C. penalize monotonicity violations when training, D. re-label samples to be monotonic before training. The four categories are better to be viewed in a structural mode with different coupling forms, such as Categories A and C are a superposition in Fig. 8(a), B in Fig. 8(d), and D in Fig. 8(c), respectively.

In [101], Fan et al. showed a study of CFS in their KDDM approach (Fig. 9). When a KD submodel (called GreenLab) was given, they used RBF networks as a DD submodel. Two models, KDDM_sup in Fig. 9(a) and KDDM_com in Fig. 9(b), were investigated. The input variables were five environmental factors and the output was the total growth yield of tomato crop. After using the real-world data of tomato growth datasets from twelve greenhouse experiments over five years in a 12-fold cross-validation testing, KDDM_com was selected because it showed a better prediction performance over KDDM_sup. In fact, KDDM_com demonstrated a better interpretation about one variable, $E$, in plant growth dynamics than that of KDDM_sup. Their study also demonstrated several advantages of KDDM approach over the conventional KD or DD models, such as predictions of yields from different types of organs even some training data were missing.

One can see that CFS is an extra challenge over the conventional modeling studies, but required to be explored systematically. We still need to define related criteria in the selection. When a performance is a main concern, some other issues may be also considered, such as learning speed, or interpretation of a model. CFS presents a new subject in AI studies when more models or tribes are merged together as the Master Algorithm.

In the conventional ML study, the common scheme for imposing constraints is the Lagrange multiplier as a standard method [47, 74, 48]. The question can be asked like “When ANNs emulate the synaptic plasticity functions of human brains, does a human brain apply the Lagrange multiplier if giving a new set of constraints”? Reviewing the image example of Fig. 3 again, if one relies on Fig. 16 for the correct recognition, we still do not know with which mathematical methods do our brains apply for describing and imposing the constraints.

In [117], Cao et al. attempted to address this question based on the “Locality Principle (LP) ” [118, 119]. LP is originally come from classical physics in the law of gravity and states that an object is mostly influenced by its immediate surroundings. In computer science, Denning and Schwartz [118] pointed out in 1972 that “Programs, to one degree or another, obey the principle of locality” and “The locality principle flows from human cognitive and coordinative behaviory.” In 2005, Denning further described that: “The locality principle found application well beyond virtual memory” and “Locality of reference is a fundamental principle of computing with many applications.” In neuroscience, LP can be supported from the knowledge: certain locations of a brain are responsible for certain functions (e.g. language, perception, etc.) [120]. Hence, Cao et al. [117] stated that “All constraints can be viewed as memory. The principle provides both time efficiency and energy efficiency, which implies that constraints are better to be imposed through a local means.” They proposed a non-Lagrange multiplier method for equality constraints, falling within a locally imposing scheme (LIS). The main idea behind the proposed method was to make local modifications to the solutions after constraints were added. The method transformed equality constraint problems into unconstrained ones and solved them by a linear approach, so that convexity of constraints was no more an issue. The numerical examples were given on solving PDEs with Dirichlet or Neumann boundary constraints. The proposed method was possible to achieve an exact satisfaction of the equality constraints, while the Lagrange multiplier method presented only approximations. They further compared the two methods numerically on a one dimensional “Sinc”, $f(x)=sin(x)/x$, with the interpolation constraints as: $f(0)=1$ and $f(\pi/2)=2/\pi$. The comparison aimed at “how to discover Lagrange multiplier method to be globally imposing scheme (GIS) or LIS?” They found the answers to be interpretation form depended. Suppose an interpretation form is given in the following expression:

where $f_{wc}(x)$ is the RBF output without constraints and $f_{m}(x)$ is the modification output after constraints are added in. From Fig. 10, one can see that $f_{m}(x)$ shows a global modification, or GIS, when using the Lagrange multiplier, but a local and smooth modification, or LIS, when using the proposed method. This interpretation form provided a locality interpretation from a “signal modification” sense. They also investigated the other interpretation form, but failed to give the clear conclusion from a “synaptic weight changes” sense.

The study in [117] is very preliminary, but shows an important direction for the future ML or AI studies by the following aspects:

• •

  Locality principle is one of the important bases for realizing time efficiency and energy efficiency of bio-inspired machines. We need to transfer principles, or high-level GCs, into mathematical methods in AI machine designs. Convolutional neural networks (CNNs) [90] and RBFs [121] show the good examples in terms of the principle, satisfying a restricted region of the receptive field in biological processes [122]. More investigations are needed to explore LP in a wider sense for designs, such as LIS on the inequality constraints, or its role in continuous learning with another important GC: no catastrophic forgetting (NCF) [123, 124].

• •

  The hypothesis and selection of LIS and/or GIS suggest a new subject in the study of brain modeling or bio-inspired machines. When LP is rooted on the classical physics, quantum mechanics might violate locality from the entanglement phenomenon [125]. The different studies have been reported about brain modeling based on quantum theory, such as quantum mind [126, 127] and quantum cognition [128]. Those studies suggested that consciousness should be a quantum-type prior, and should be treated with non-classical methods. The subject shows that LP or nonlocality will be a main issue for realizing a brain-inspired machine, and GCs should be treated according to their principle behind.

In [2], a GC was given first in a linguistic form, as shown by the GCs example in Table 1. This example describes the Mackey-Glass dynamic process, shown in eq. (5). The linguistic GC (or hypothesis) was finally transformed into a mathematic form as

For the given time series dataset with $\tau=17$ and $b=0.1$, Hu et al. [2] used their GCNN model for the prediction and also obtained the solutions $\hat{\tau}=17$ and $\hat{b}=0.1096$.

In the same paper, Hu et al. also used GCNN as a hypothesis tool to discover the hidden property of data in approximating “Sinc” function, eq. (9). For the given GC, a GCNN model exhibited an abrupt change in the output $f$ when $x=y=0$. They suggested the hypothesis of “removable singularity” for the abrupt change, and confirmed it from the data and analysis procedures. Their study showed that ML models can be a tool for identifying unknown parameters and hidden properties of physical systems.

Recently, Alaa and van der Schaar [34] proposed a novel and encouraging direction to extract mathematic GCs in form of analytic functionals from ANN models. In their study, a black-box ANN model was described by $f(\mathbf{x})$. They formed a symbolic metamodel $g(\mathbf{x})$ using Meijer $G$-functions. The class of Meijer $G$-functions includes a wide spectrum of common functions used in modeling, such as, polynomial, exponential, logarithmic, trigonometric, and Bessel functions. Moreover, Meijer $G$-functions are analytic and closed-form functions, even for their differential forms. By minimizing a “metamodeling loss” $l(g(\mathbf{x}),f(\mathbf{x}))$, they obtained $g(\mathbf{x})$ with a parameterized representation of symbolic expressions. They conducted numerical experiments on four different functions as the given GCs, that is, exponential, rational, Bessel and sinusoidal functions, respectively. Their approach was able to figure out the first three functional forms among the four functions.

In Fig. 1, we show that functional space is among the primary concerns in ML or AI. When the true functional form is unknown to the system investigated, modelers will involve a subject defined below:

FFS has received little attention in AI communities. For simplification, most of the existing AI models directly apply a preferred functional without involving FFS. If considering AI to be a discovering tool, FFS should be considered first in modeling, particularly to complex processes, say, in economics [150]. In view of system identification [151, 152], FFS will be more basic and difficult than parameter estimations in nonlinear modeling. The study in [34] shows an important “gateway” in FFS, as well as in obtaining more fundamental knowledge, functional forms, about physical systems. Similar to the subject of CFS, FFS shows another open problem requiring a systematic study for ML or AI communities.

Bellman and Åström [153] proposed the definition of structural identifiability (SI) in modeling. The term “structural” means the internal structure of a model, so that SI is independent of the data applied. Because any ML model with a finite set of parameters can be viewed as a “parameter learning machine” [11], Ran and Hu adopted the notion of parameter identifiability to be the theoretical uniqueness of model parameters in statistical learning [154]. Yang et al. [155] showed an example why parameter (or structural) identifiability becomes a prerequisite before estimating a physical parameter in GCNN model (Fig. 8(b)) in a form of:

where damping coefficient $\alpha$ ($>0$) is a single physical parameter in the KD submodel $f_{k}(x,\mathbf{\theta_{k}})$. They demonstrated that $\alpha$ is unidentifiable if $f_{d}(x,\mathbf{\theta_{d}})$ is a RBF submodel, which suggests no guarantee of convergence to the true value of $\alpha$. However, $\alpha$ becomes identifiable if a sigmoidal feed-forward network (FFNs) submodel is applied. In a review paper [154], Ran and Hu presented the reasons of identifiability study in ML models:

“Identifiability analysis is important not only for models whose parameters have physically interpretable meaning, but also for models whose parameters have no physical implications, because identifiability has a significant influence on many aspects of a learning problem, such as estimation theory, hypothesis testing, model selection, learning algorithm, learning dynamic, and Bayesian inference.”

For example, Amari et al. [156] showed that FNNs or Gaussian mixture models (GMMs) will exhibit very slow dynamics of learning due to the nonidentifiability of parameters from plateaus of neuromanifold. Hence, parameter identifiability provides mathematical understanding, or explainability, about learning machines in terms of their parameter spaces involved. For details about parameter identifiability in ML, see [157, 67, 154] and the references therein.

Data visualization (DV) is a graphic representation of data for modelers/users to understand the features of the data. In ML studies, DV is one of the most common techniques in modeling so that modelers are able to gain the related GCs for the design guidelines as well as for the interpretation to the response of models [158]. This technique can extend to the subjects of information visualization (IV) [159] or visual analytics (VA) [160]. Some DV tools are able to present knowledge from data in a mathematic form, such as histogram, heatmap, etc. We present an example below to show that this is not a case in general. DV tools usually exhibit knowledge in a form of GCs, which require users to seek and abstract. Constraint mathematization will be another challenge in abstractions of knowledge.

When high-dimensional data are difficult to interpret, researchers developed many approaches for viewing their intrinsic structures in a low-dimensional space [161]. In [162], van der Maaten and Hinton proposed an approach called t-SNE (t-Distributed Stochastic Neighbour Embedding) for visualization of high-dimensional data in a 2-D or 3-D space. Different from a PCA approach, t-SNE is a nonlinear dimensionality reduction technique. For the handwritten digits 0 to 9 in the MNIST dataset, they showed the 2-D visualization results with t-SNE (Fig. 11). The raw data is given in high dimensions with a size of 784 ($28\times 28$ pixels). The t-SNE plot of Fig. 11 clearly demonstrates the hidden patterns of clusters in the high dimensional data. The patterns present a rather typical form of GCs instead of CCs. Modelers need to summarize them in form of either natural languages or mathematical descriptions, such as visual outliers [163] in classifications, or semantic emergence, shift, or death in dynamic word embedding [164] from t-SNE plots.

The most challenge in visualization is to develop/seek a DV tool suitable for revealing intrinsic properties, or theoretical understanding, about data. One good example is a study by Shwartz-Ziv and Tishby [165] in proposing a DV tool, called information plane. This tool presented a novel visualization of data from information-theoretic understanding, and was used for explaining the structural properties and leaning properties of DNNs.

Model visualization (MV) is a graphic representation of ML/AI models about their architectures, learning processes, decision patterns, or related entities for modelers/users to understand the properties of the models. MV is another means to foster insights into models, and it may often involve techniques from DV, such as learning trajectories in optimizations [166], or a heatmap for explaining convolutional neural networks (CNNs) [167, 168].

If regardless of a common representation of architecture diagrams of models, it was reported that Hinton diagrams [169] was among the first tools in visualizing ANN models [170]. This tool was designed for viewing connection strengths in ANNs, using color to denote sign and area to denote magnitude. With the help of Hinton diagram, Wu et al. [171] gained the knowledge about unimportant weights for pruning ANN architectures. In 1992, Craven and Shavlik [170] reviewed a number of visualization techniques for understanding decision-making processes of ANNs, such as Hinton diagrams [169], bond diagram and Trajectory diagrams [172], etc. They also developed a tool, called Lascaux, for visualizing ANNs. Graphic symbols were used to show the connections and neurons with activation and error signals, so that modelers were able to gain some insight into the learning behavior of ANNs. In the early study of MV, some researchers proposed a study of “Illuminating or opening the black box” of ANN models [173, 174]. In [173], Olden and Jackson applied neural interpretation diagram (NID), Garson’s algorithm [175], and sensitivity analysis, so that variable selections can be understood by visual perception of modelers. In [174], Tzeng and Ma used a new MV for knowing the working architectures of ANNs when the voxel of an image was corresponding to the brain material or not. The different architectures provided the relational knowledge between the important variables and voxel types.

When deep learning (DL) models are emerged, more MV studies are reported on understanding the inner-workings of this type of black-box models [176, 177]. In [178], Liu et al. developed a MV tool for visualizing the deep CNNs, called CNNVis. Fig. 12 shows a CNN model including four groups of convolutional layers and two fully connected layers. The model consisted of thousands of neurons and millions of connections in its architecture. Using CNNVis to simplify large graphs, they were able to show representations in clusters for better and fast understanding about pattern knowledge of CNN models. They conducted case studies on the influence of network architectures and diagnosis of a failed training process to demonstrate the benefits of using CNNVis. In a different GC form, Zhang et al. [179] applied decision trees in a semantic form to interpret CNNs models. For the other DL models, one can refer to the studies on recurrent neural networks (RNNs) [180, 181] and reinforcement learning (RL) [182, 183]. Some review papers [184, 177, 185] appeared recently about visualization of DL models. In general, MV techniques provide intuitive GCs in an unstructured form, and present a human-in-the-loop interactive of AI [185], such as to modify models or to adjust learning processes [160, 186].

In general, we can say that graphical models (GMs), using graphic representations, are the best and direct means to describe worlds, either physical or cyber one. The main reason is from the facts below: (i) most objects are distinguished from their geometries, and (ii) they may be linked internally by their own entities or externally to each others. For every objects or data, their underlying graph structures are the most important knowledge we need to explore [188]. Buntine [189] pointed out that “Probabilistic graphical models are a unified qualitative and quantitative framework for representing and reasoning with probabilities and independencies”, and “are an attractive modeling tool for knowledge discovery”. We will present two sets of studies below to show that the knowledge discovered in GMs is often given in a form of GCs, instead of CCs.

In [187], Zhang et al. developed a tool, called ExpressGNN, by combing Markov logic networks (MLNs) [99] and graph neural networks (GNNs). The motivation was come from the facts that MLNs are computationally intractable from an NP-complete problem and the data-driven GNNs are unable to leverage the domain knowledge. Fig. 13 shows the working principle of ExpressGNN, in which knowledge graph (KG) is a knowledge base representing a collection of interlinked descriptions of entities. ExpressGNN scales MLN inference to KG problems and applies GNN for variational inference. With the GNN controllable embeddings, ExpressGNN were able to achieves more efficiency than MLNs in logical reasoning as well as a balance between expressiveness and simplicity of the model. In the numerical investigations, they demonstrated that ExpressGNN was able to fulfill zero-shot learning (ZSL). Fig. 14 illustrates an example [190] about ZLS, which is a learning to recognize unseen visual classes with some semantic information (or GCs) given. The term “zero-shot” means no instance to be given in the training dataset. ZLS is a kind of GCL because semantic information and/or unseen classes may be ill-defined or incompletely described in a form of GCs. Semantic gap may be involved for GMs, both in encoding and decoding of GCs, such as discovering unseen classes.

Another example is about studying co-evolving financial time-serious problems from GMs. In [191], Bai et al. proposed a tool, called EDTWK (Entropic Dynamic Time Warping Kernels), for time-varying financial networks. They computed the commute time matrix on each of the network structures for satisfying a GC as “the financial crises are usually caused by a set of the most mutually correlated stocks while having less uncertainty [192, 191]”. Based on the commute time matrix, a dominant correlated stock set was identified for processing. They conducted numerical investigations from the New York Stock Exchange (NYSE) dataset. For the given data, EDTWK was formed as a family of time-varying financial network with a fixed number of 347 vertices from 347 stocks and varying edge weights for the 5976 trading days. They used EDTWK to classify the time-varying financial networks into corresponding stages of each financial crisis period. Five sets of the crisis periods were studied, that is, Black Monday, Dot-com Bubble, 1997 Asia Financial Crisis, Newcentury Financial Bankruptcy, and Lehman Crisis. EDTWK was able to detect them from the given data and to characterize different stages in time-varying financial network evolutions. Some crisis events were shown in 3-D embedding, such as the two plots from two events in Fig. 15. One can see that the main challenge is still about how to discover and abstract GCs from the model and its visualization, either for modelers or for machines.

The examples given above indicate that, in AI modeling, it is a promising direction of merging other models with GMs, or utilizing graphic representations as much as possible. The studies in natural language processing (NLP) [193, 194, 195] have demonstrated that GMs can process natural language and have achieved much better performance than the conventional NLP models. Due to a transparent feature of GMs [67], more studies have been reported, such as on GNNs [196, 100, 197] and references therein.