An efficient optimization based microstructure reconstruction approach with multiple loss functions

Anindya Bhaduri Ashwini Gupta Audrey Olivier Lori Graham-Brady¹¹1Corresponding author. Email address: [email protected] Department of Civil Engineering, Johns Hopkins University, Baltimore, MD, USA Columbia University, New York, NY, USA

Abstract

Stochastic microstructure reconstruction involves digital generation of microstructures that match key statistics and characteristics of a (set of) target microstructure(s). This process enables computational analyses on ensembles of microstructures without having to perform exhaustive and costly experimental characterizations. Statistical functions-based and deep learning-based methods are among the stochastic microstructure reconstruction approaches applicable to a wide range of material systems. In this paper, we integrate statistical descriptors as well as feature maps from a pre-trained deep neural network into an overall loss function for an optimization based reconstruction procedure. This helps us to achieve significant computational efficiency in reconstructing microstructures that retain the critically important physical properties of the target microstructure. A numerical example for the microstructure reconstruction of bi-phase random porous ceramic material demonstrates the efficiency of the proposed methodology. We further perform a detailed finite element analysis (FEA) of the reconstructed microstructures to calculate effective elastic modulus, effective thermal conductivity and effective hydraulic conductivity, in order to analyse the algorithm’s capacity to capture the variability of these material properties with respect to those of the target microstructure. This method provides an economic, efficient and easy-to-use approach for reconstructing random multiphase materials in 2D which has potential to be extended to 3D structures.

keywords:

microstructure reconstruction; transfer learning; Gram matrix; total variation; 2-point probability; stochastic simulation; two-phase materials

1 Introduction

The discovery and development of new materials with targeted properties has always been a central theme for computational material science research, which requires a fundamental understanding of materials behavior and properties across different scales. Stochastic microstructure reconstruction is an important piece of that challenging puzzle. It enables computational generation of microstructures that match the statistically equivalent characteristic(s) of a (set of) target microstructure(s), avoiding the need for exhaustive and costly physical microstructure characterization of every sample to be evaluated.
There exists a number of different types of microstructure reconstruction approaches in the current literature. Reconstruction via statistical functions [1, 2, 3] is a popular technique applicable to a wide range of materials such as, particulate structures [2, 4], chalk [5], soil [6], sandstone [7, 8].The reconstruction is usually done via a stochastic optimization method, known as Yeong and Torquato (YT) method [9, 10] and tries to ensure that the chosen statistical functions of the target microstructure match closely with that of the reconstructed microstructure. However, if the target image is anisotropic or multiphase or large in size, the convergence of the YT method is especially slow which makes the procedure computationally expensive. Physical descriptor-based approaches [11, 12, 13], on the other hand, involve a relatively cheaper optimization procedure which tries to match the physically meaningful descriptors of the target microstructure to that of the reconstructed one. However, their application is limited to the microstructure reconstruction of crystalline and particulate structures (regular geometries) and not to material systems with irregular geometries. Spectral density function (SDF)-based approaches aim to reconstruct statistically equivalent microstructure by generating SDF representations of the target microstructure and matching it to the reconstructed one. The methods [14, 15, 16, 17] are mostly analytical in nature and hence much faster than the optimization-based reconstruction approaches but their application is restricted to isotropic binary materials. Another class of microstructure reconstruction methods involves using deep learning [18, 19], a machine learning approach that can be used amongst other tasks for surrogate modeling [20, 21, 22, 23, 24, 25, 26] and thus has been implemented successfully for a wide range of classification and regression tasks. Deep learning approaches, in particular convolutional neural networks, are particularly well-suited to handle image data, and have thus received attention lately from the materials research community to process microstructures image data for a variety of tasks [27, 28]. Deep learning approaches for reconstructions are of two different types: material-system-dependent and material-system-independent approaches. Material-system dependent approaches include work by Cang et al. [29] which uses a convolutional deep belief network [30] and work by Li et al. [31] where a Generative Adversarial Network (GAN) [32] model is employed. These approaches train the weights of the network with images specific to a material system and thus need to be retrained for a new material system. Alternatively, transfer learning approaches are material-system-independent and does not require training weights with a set of materials data. Instead, deep learning models, pre-trained using benchmark datasets in the field of computer vision, are used to generate microstructure reconstructions. However, it is noted that, given the fixed pretrained weights, there still may be a need for hyperparameter tuning to get the most optimal results for a given material data set. The work by Li et al. [33] is one such approach where a deep convolutional network VGG-19 [34] pretrained on ImageNet [35] dataset has been used for the reconstruction based on a single given target microstructure. A feature-matching optimization has been performed using a Gram-matrix loss function to generate statistically equivalent microstructures. The reconstruction approach presented in this paper is derived from the work by Li et al [33].
In this paper, we propose a modified version of the existing transfer learning approach [33]. The modifications include: a) different combinations of Gram matrix layers, b) a weighted combination of the gram matrix (GM) loss components, c) elimination of the potentially costly simulated-annealing based volume fraction matching process, d) inclusion of microstructural descriptor metrics in the overall loss function in the main optimization step. The modifications aim to enhance the efficiency, accuracy as well as interpretability of the proposed modified approach compared to that of the existing approach. It is noted here that the performance of the proposed approach is assessed via a statistical analysis of both the physics-based descriptors of the microstructure as well as the Finite Element Analysis (FEA) simulated averaged material properties. The analysis is demonstrated using a true microstructure dataset consisting of $185$ images of size $560$ pixels $\times$ $902$ pixels, corresponding to $185$ successive slices of a porous ceramic microstructure measured by x-ray tomography similar to those found in [36, 37]. The target microstructures are described in section $2$ . Section $3$ discusses the denoising of the original target microstructures and compares the statistical properties between the original and the denoised versions. Section $4$ explains the six microstructural descriptors considered in this study to assess the reconstruction quality. The microstructure reconstruction procedure with different loss function combination and different slices (original and denoised) as the target image is discussed in details in section $5$ . In section $6$ , FEA simulated material properties are compared between the original and the reconstructed images. Section $7$ provides a discussion of the additional advantages of the proposed approach. Section $8$ provides the conclusions.

2 Target microstructures

The target microstructure is that of a two-phase porous ceramic material and there are images of $185$ successive slices of size $560$ pixels $\times$ $902$ pixels. Thus a binary indicator function $I_{(1)}$ : $(X,Y)\in\Omega\rightarrow\{0,1\}$ represents the microstructure and is defined as

I_{(1)}(X,Y)=\begin{cases}1,&\text{if }(X,Y)\in\Omega^{(1)}\\ 0,&\text{if }(X,Y)\in\Omega^{(0)}\end{cases}

where $\Omega$ is the material domain, $\Omega^{(0)}\subset\Omega$ is the black solid phase domain and $\Omega^{(1)}\subset\Omega$ is the white porous phase domain such that $\Omega^{(0)}+\Omega^{(1)}=\Omega$ and $\Omega^{(0)}\cap\Omega^{(1)}=\emptyset$ . This is discretized in a computational setting by an array of (0,1) values. The slice $1$ microstructure is shown in figure 1.

Refer to caption — Figure 1: Target microstructure (slice $\#$ 1)

3 Denoising of target microstructure

The need for denoising of the original target microstructure arises from the fact that spurious inclusions of only one or a few pixels appear in the image. These spurious pixels significantly complicate finite element (FE) analyses used to calculate effective properties. While denoising enables coarser discretization of the material domain and more efficient FE analysis, too much denoising can lead to significant changes in the local microstructure. Keeping this in mind, all cluster sizes for the black solid phase less than $10$ pixels are converted to the white porous phase. Figure 2 shows a comparison of the original and the corresponding denoised microstructures for slices $\#1$ , $\#90$ and $\#185$ .

4 Microstructural descriptors

Microstructural descriptors are considered key in determining the effective physical properties of random heterogeneous materials [38]. Thus, it is important to match the microstructural descriptors of the reconstructed microstructure with that of the original microstructure. In this study, six such descriptors are considered: porosity, correlation length, specific boundary length, mean pore size, $90^{th}$ -percentile pore size and $97^{th}$ -percentile pore size. Each of these descriptors are described below and some are shown in figure 3.
Porosity is the fraction of the porous phase in the microstructure given by:

\phi=\frac{\sum_{i=1}^{N_{x}}\sum_{j=1}^{N_{y}}I_{(1)}(x_{i},y_{j})}{N_{x}N_{y}}

(1)

where $(x_{i},y_{j})$ is any pixel location in the microstructural domain $\Omega$ , while $N_{x}$ and $N_{y}$ are the number of pixels along the horizontal and the vertical directions respectively. Figure 4(a) shows the porosity of each of the 185 slices of the material. Denoising slightly increases porosity since small regions of the black solid phase are replaced by the white porous phase.
While the volume fraction provides information about the microstructural phase at individual points, it does not provide information about the spatial patterns of the microstructure that can be explained to some extent by the two-point probability function. The two-point probability function $S_{2}^{(1)}(\Delta_{l},\Delta_{m})$ calculates the probability that any two points $(x_{i},y_{j})$ and $(x_{i+l},y_{j+m})$ with a separation vector $(\Delta_{l},\Delta_{m})$ fall in the white porous phase and is given by:

	$\displaystyle S_{2}^{(1)}(\Delta_{l},\Delta_{m})$	$\displaystyle=P\big{[}I_{(1)}(x_{i},y_{j})=1,I_{(1)}(x_{i+l},y_{j+m})=1\big{]}$
		$\displaystyle=\frac{\sum_{i=1}^{N_{x}-l}\sum_{j=1}^{N_{y}-m}I_{(1)}(x_{i},y_{j})I_{(1)}(x_{i+l},y_{j+m})}{(N_{x}-l)(N_{y}-m)}$		(2)

It thus provides a measure of how the two end points of a vector (denoted by a blue solid line in figure 3) in a phase (in this case, the white porous phase) are correlated. It is noted that the above description of the two-point probability function similarly applies for the solid phase. In the above definition, it is assumed that the material is spatially stationary (or statistically homogeneous), which may not be valid for materials in which the microstructural statistics vary spatially (e.g., in a functionally graded material). Correlation length is one scalar measure associated with the two-point probability function, which is calculated by integrating the normalized two-point probability function over the entire domain [39] and then taking its square root:

\alpha=\sqrt{\frac{S_{2}^{(1)}(\Delta_{l},\Delta_{m})-\phi^{2}}{\phi-\phi^{2}}}

(3)

Figure 4(b) shows the correlation length of each of the 185 slices of the material. It is noted that denoising slightly increases correlation length because it removes small correlation length inclusions.

Specific boundary length is the cumulative length of the boundary of separation between the black solid phase and the white porous phase, normalized by the total microstructural area. The cumulative boundary length denoted by red solid lines in figure 3 is obtained by generating the corresponding morphological gradient of the microstructure by taking the difference of its dilation and erosion. The morphological gradient of the microstructure is a matrix of the same size as the microstructural image with non-zero valued pixels along the boundaries of the two phases and zero-valued pixels elsewhere. The cumulative boundary length is thus obtained by counting all the non-zero pixels in the morphological gradient image. Figure 4(c) shows the specific boundary length of each of the 185 slices of the material for the original and denoised cases. It is seen that denoising slightly decreases the specific boundary length because the replacement of small clusters of black solid phase by white porous pixels leads to a decrease in the cumulative boundary length.
The pore-size probability density function $P(\delta)$ is used to characterize the void or “pore” space in porous media [40]. The function $P(\delta)$ for isotropic media is defined such that $P(\delta)d\delta$ is the probability that a randomly chosen point in the porous phase domain $\Omega^{(1)}$ lies at a distance between $\delta$ and $\delta+d\delta$ from the nearest point on the pore-solid interface. Thus, $\int_{0}^{\infty}P(\delta)d\delta=1$ . The mean pore size is defined as:

\langle\delta\rangle=\int_{0}^{\infty}\delta P(\delta)d\delta

(4)

The associated complementary cumulative distribution function $F(\delta)$ is the fraction of pore space that has a pore radius larger than $\delta$ and is given by:

F(\delta)=\int_{\delta}^{\infty}P(\delta)d\delta

(5)

$90^{th}$ -percentile pore size is defined as the pore size such that $90\%$ of the pore space has a pore radius less than or equal to that pore size, i.e., $F(\delta)=0.10$ . $97^{th}$ -percentile pore size, similarly, is defined as the pore size such that $97\%$ of the pore space has a pore radius less than or equal to that pore size, i.e., $F(\delta)=0.03$ . Figures 4(d), 4(e) and 4(f) show the variation of the mean pore size, the $90^{th}$ -percentile pore size and the $97^{th}$ -percentile pore size, respectively, across all 185 slices for the original and their corresponding denoised microstructures. It is seen that denoising has very little effect on these three statistical properties of the microstructure.

5 Microstructure reconstruction

In this section, the reconstruction procedure and the corresponding results are discussed. The algorithm is an optimization based procedure and different loss function combinations are implemented with an optimal set of hyperparameters for the reconstruction of original slice $\#1$ which is discussed in sections 5.1, 5.2 and 5.3. Section 5.4 uses the same set of hyperparameters and compares the reconstruction results of original slice $\#1$ with that of original slices $\#90$ and $\#185$ . In section 5.5, reconstruction is performed on all the original $185$ slices and their denoised versions with different initializations of the reconstructed image, and the corresponding results are provided. It is noted here that the initial reconstructed image is a white noise color ( $3$ -channels) image generated by random sampling. Thus the random number generator seed can be changed to generate different initial noisy images that eventually lead to statistically equivalent but distinct reconstructed microstructures. The reconstruction algorithm involves conversion of the $3$ -channels representation of the reconstructed color image to the single channel 2-phase representation. This is achieved by a binary $(0/1)$ classification of the 3D color image pixel coordinates based on their euclidean distances from the 3D black pixel coordinate $[0,0,0]$ and the 3D white pixel coordinate $[255,255,255]$ .

5.1 Reconstruction with Gram matrix loss

In this section, a transfer learning based pattern-matching optimization is performed using a pre-trained VGG-19 network shown in figure 5. VGG-19 is a deep convolutional neural network and in total has $16$ convolutional layers and $5$ max pooling layers, divided into $5$ blocks. The original network, in addition, has $3$ fully connected layers at the end which are excluded here. It is noted that $3$ -channels representation is a requirement for the input image to this network [33]. Thus, the $2$ -phase single channel representation of the original microstructure is converted to a $3$ -channels representation by stacking $3$ duplicates of the single channel image. Any intermediate output from layer operation in a convolutional neural network is called a feature map. Convolutional layers [41, 42] consists of one or more kernels (matrix of trainable weights) that convolve the input (dot product between the kernel-sized patch of the input and kernel) and pass it on to the next layer. Each convolution layer is followed by a nonlinear activation ReLU (rectified linear unit) [43, 44] layer which induces non-linearity into the network model. Max pooling [45, 46] uses a max filter for down sampling by selecting the maximum value of the non-intersecting kernel-sized patches of the feature map. Max pooling, in essence, extracts the most prominent features of the previous feature map.
A study has been performed to select the optimal number and combination of VGG-19 network layers for the most accurate reconstruction with respect to the Gram matrix (GM) loss calculation. Gram matrix [47] is a measure of texture in an image and the gram matrix loss here is a measure of the difference between the textures of different intermediate image outputs from the VGG-19 network for the original and the reconstructed microstructure respectively. Following the same notations as in [33], let $F_{pr}^{i}$ and $\tilde{F}_{pr}^{i}$ denote the feature maps of the $p^{th}$ filter at position $r$ in layer $i$ for the original and the reconstructed microstructures, respectively. The Gram matrix $G_{pq}^{i}$ of the original microstructure and $\tilde{G}_{pq}^{i}$ of the reconstructed microstructure, in layer $i$ , is thus defined by the following inner product:

G_{pq}^{i}=\sum_{r}F_{pr}^{i}F_{qr}^{i}

(6)

\tilde{G}_{pq}^{i}=\sum_{r}\tilde{F}_{pr}^{i}\tilde{F}_{qr}^{i}

(7)

The Gram matrix loss for the layer $i$ is given by:

E_{i}=\frac{1}{4N_{i}^{2}M_{i}^{2}}\sum_{j,k}(G_{jk}^{i}-\tilde{G}_{jk}^{i})^{2}

(8)

where $N_{i}$ is the number of filters and $M_{i}$ is the size of the vectorized feature maps in layer $i$ . The total Gram matrix loss across the selected layers is thus given by:

L_{G}=\sum_{i}w_{i}E_{i}

(9)

where the weight of each layer $i$ is given by $w_{i}=\frac{min_{i}\sum_{j,k}(G_{jk}^{i})^{2}}{\sum_{j,k}(G_{jk}^{i})^{2}}$ .

The first convolutional layers of each of the 5 blocks of the VGG-19 network as shown in figure 5 is found to be most optimal. The overall optimization framework is shown in figure 6. The reconstruction algorithm was run for $50$ different initializations of the reconstructed microstructure.

Figure 7 compares the two-point probability function and the pore size cumulative density function (CDF) for the original slice $\#1$ microstructure and the reconstructed image corresponding to one of the $50$ initializations.

Figure 8 shows the reconstructed image corresponding to the original slice $\#1$ image. Although the statistical property estimates of the reconstructed image are close to that of the target slice $\#1$ , on visual inspection it is found that spurious checkerboard patterns are present in the reconstructions as seen in figure 8.

5.2 Reconstruction with GM and TV loss

To address the checkerboard patterns, total variation (TV) loss between the target image and the reconstructed image is included in the optimization framework as shown in figure 9. The total variation for an image is defined as the sum of the absolute differences in the pixel values between neighboring pixels in that image. It is a measure of noise in an image that is sensitive to features like checkerboard patterns. The total variation loss is the anisotropic version of the total variation norm [48] and is given by:

L_{V}=\sum_{k}\sum_{i,j}|y_{i+1,j,k}-y_{i,j,k}|+|y_{i,j+1,k}-y_{i,j,k}|

(10)

where $y_{i,j,k}$ is the ${(i,j,k)}$ -th pixel value of the 3D representation of the reconstructed image. Figure 10 shows the reconstructed image corresponding to one of the initializations, and it is evident that the inclusion of the total variation loss eliminates the checkerboard patterns found in the reconstructed images in section 5.1. It is also noted that similar observations are found in all the other initialization cases. The total loss $L_{T}$ in this case is given by:

L_{T}=w_{G}L_{G}+w_{V}L_{V}

(11)

where $w_{G}=1$ and $w_{V}=1$ e $-6$ are the weight combinations that seem to produce the best results using ‘trial and error’ approach. Table 1 shows the mean error in the statistical properties of interest calculated over $50$ different initialization cases for the reconstruction with the combined “GM+TV” loss and is compared to the corresponding results with only the GM loss discussed in section 5.1. It is seen that the mean error for almost all the statistical properties improves for the “GM+TV” loss case compared to the GM loss case. Although the error in the porosity is very low in both cases, the porosity matches slightly better on an average for the GM loss case.

Table 1: Error in reconstruction with GM loss and “GM+TV” loss

	GM loss	GM+TV loss
Mean error in porosity	0.0027	0.0048
Mean error in correlation length	0.2923	0.0580
Mean error in specific boundary length	0.0144	0.0050
Mean error in mean pore size	0.1225	0.0228
Mean error in 90-th percentile pore size	0.3846	0.0947
Mean error in 97-th percentile pore size	0.8641	0.1953

5.3 Reconstruction with GM and TV and 2PP loss

In this section, we try to improve the reconstruction quality further by including another loss function in the optimization, the two-point probability (2PP) loss $L_{P}$ . This loss is defined as the mean squared error between the 2-point probability functions [refer to eq. (4)] of the original and the reconstructed microstructures. The two-point probability is sensitive to the spatial distribution of a given phase. The overall optimization framework is shown in figure 11. The total loss $L_{T}$ in this case is given by:

L_{T}=w_{G}L_{G}+w_{V}L_{V}+w_{P}L_{P}

(12)

where $w_{G}=1$ , $w_{V}=1$ e $-6$ and $w_{P}=1$ e $8$ are the weight combinations that seem to produce the best results using ‘trial and error’ approach. It is noted that a loss function corresponding to any other physical property of interest can be added to the optimization framework in a similar fashion. Figure 12 shows the box plot of the errors in the statistical properties of interest corresponding to each of the loss function cases: “GM”, “GM+TV” and “GM+TV+2PP” for the reconstruction of original slice $\#1$ . The errors are calculated over $50$ different initializations of the reconstructed image.

It is clear from the box plots that, with the exception of the very small porosity errors, all the other statistical properties match more accurately for the “GM+TV” and “GM+TV+2PP” loss cases compared to the “GM” loss case. However, the performance of the “GM+TV” and “GM+TV+2PP” loss cases are very similar to each other and the addition of the two-point probability loss function to the optimization framework does not improve the reconstruction results significantly, which is evident from the mean and coefficient of variation (CV) values of the absolute error in the properties shown in table 2.

Table 2: Error in reconstruction with “GM+TV” loss and “GM+TV+2PP” loss

	GM+TV loss		GM+TV+2PP loss
	Mean	CV	Mean	CV
Error in porosity	0.0048	0.6402	0.0046	0.6169
Error in correlation length	0.0580	0.6698	0.0615	0.6437
Error in specific boundary length	0.0050	0.1587	0.0050	0.1867
Error in mean pore size	0.0228	0.7957	0.0247	0.7321
Error in 90-th percentile pore size	0.0947	0.6825	0.0992	0.6295
Error in 97-th percentile pore size	0.1953	0.6432	0.1873	0.6288

5.4 Performance comparison with other slices

In the previous sections, reconstructions were performed with original slice $\#1$ as the target microstructure. In this section, with the same hyperparameter settings for the “GM+TV” and “GM+TV+2PP” loss cases as for slice $\#1$ -based reconstruction, the reconstruction algorithms are run with original slices $\#90$ and $\#185$ as the target microstructure over $50$ initializations. These runs are performed to verify whether the optimal hyperparameter settings for the slice $\#1$ -based reconstruction works well for the reconstruction on other slices too. Slices $\#90$ and $\#185$ are chosen as representative slices for that study. Figure 13 shows the performance comparison in the statistical properties for the “GM+TV” and “GM+TV+2PP” loss cases when the target microstructures are slices $\#1$ , $\#90$ and $\#185$ respectively. It is seen that the relative absolute errors in the statistical properties are in similar orders of magnitude across the three different slices for either of the loss cases. For each of the three target slice-based reconstructions, the reconstruction quality for both the loss cases are very similar to each other.

5.5 Reconstruction on all slices

In this section, reconstruction is performed by considering all $185$ original microstructure slices and their corresponding denoised versions individually as the target images. Three different initializations of the reconstructed image (leading to three distinct statistically equivalent images) are chosen for each of target image. Therefore, overall there are 555 reconstructions cases and their results are shown in figure 14. Figure 14 shows the comparison of the error in the statistical properties after reconstructions with the “GM+TV” and “GM+TV+2PP” loss cases, once with the original slices as the target microstructures and in the other instance, with the denoised slices as the target images. It is found that the comparison of almost all statistical properties are very similar to each other for all four reconstruction scenarios. The only exception is for the specific boundary length values where the overall error for the reconstruction cases with the denoised target slices are lower than that with the original target slices.

6 FE analysis based material properties

In this section, quality of the reconstructions are assessed in terms of a few averaged material properties of the porous ceramic material. The material properties of interest are the effective Young’s modulus ( $\bar{E}$ ), effective thermal conductivity ( $\bar{\lambda}$ ) and effective hydraulic conductivity ( $\bar{K}$ ). After an image is reconstructed based on a target image, the properties of the reconstructed microstructure are simulated using the commercial FE software ABAQUS [49]. $560\times 902$ elements (each pixel assigned an element) are used to model each microstructure in 2D.

A static linear elastic analysis is performed to calculate the effective Young’s modulus by subjecting the finite element model of the microstructure to a tensile load as shown in figure 15. The macroscopic stress $\bar{\sigma}_{yy}$ in the vertical direction (direction of applied displacement) is calculated [50] by summing all the nodal reaction forces on the top face along the vertical direction and dividing it by the cross sectional area $A$ of the face:

\bar{\sigma}_{yy}=\frac{\sum_{i}^{N}F^{R}_{i}}{A}

(13)

where $F^{R}_{i}$ is the reaction force on node $i$ along the vertical direction, $N$ is the number of nodes on the surface where displacement is applied. The effective Young’s modulus is then obtained using:

\bar{E}=\frac{\bar{\sigma}_{yy}}{\bar{\epsilon}_{yy}}

(14)

where macroscopic strain $\bar{\epsilon}_{yy}=\frac{\delta_{y}}{L}$ and $L$ is the edge length along the vertical direction.

The effective thermal conductivity is obtained by applying a constant temperature difference between the top and bottom faces of the finite element model of the microstructure [51] as shown in figure 16. The macroscopic heat flux density $\bar{q}_{y}$ is calculated by summing the nodal heat fluxes on the top face. The effective thermal conductivity is then calculated using Fourier’s law of heat conduction [52]:

\bar{\lambda}=\bar{q}_{y}\frac{L}{\Delta T}

(15)

where $\Delta T=T_{2}-T_{1}$ is the temperature difference between the top and bottom face, and $L$ is the length of the edge along the vertical direction.

The effective hydraulic conductivity for a laminar fluid flow through the porous medium of the material is simulated by specifying the hydraulic head between the top and bottom faces of the finite element model of the microstructure and then calculated from Darcy’s equation as shown in figure 17. The macroscopic specific discharge $\bar{q}_{y}$ is calculated by summing up the nodal specific discharge values on the top face. The effective hydraulic conductivity is then calculated using Darcy’s law of fluid flow [53]:

\bar{K}=\bar{q}_{y}\frac{L}{\Delta h}

(16)

where $\Delta h=h_{2}-h_{1}$ is the hydraulic head between the top and bottom face, and $L$ is the length of the edge along the vertical direction.

Figure 18 shows the boxplot of the relative absolute errors in the effective modulus, conductivity and hydraulic conductivity of the reconstructed microstructures with respect to the corresponding target microstructures. It is seen in figure 18(a) that the effective Young’s modulus error values for all the cases are in an acceptable range. The reconstruction quality is similar between the two loss function cases when the original images are used as target slices. However, when the denoised images are used, the “GM+TV” loss case performs better than the “GM+TV+2PP” case. The error values corresponding to the effective thermal conductivity in figure 18(b) and the effective hydraulic conductivity in figure 18(c) are significantly higher than that of the elastic modulus. This suggests that the reconstruction algorithm is not able to capture these properties efficiently. This can be attributed to the fact that the effective modulus is a function of the overall pattern of the microstructure, i.e., the relative arrangement of the black solid and the white porous phase, which is well matched in the original and the reconstructed microstructure. However, the effective thermal conductivity and hydraulic conductivity depend significantly on localizations in the microstructures that enable heat flow and fluid flow respectively. The goal of the reconstruction algorithm is to achieve statistical equivalence by matching the pattern and statistics of the original and reconstructed microstructure across the entire image domain. Thus, even with a statistically equivalent reconstructed image, differences in small localized regions can lead to significant differences in the heat and fluid flow channels, resulting in very different effective thermal and hydraulic conductivity. Overall, if the error values are compared among all the four reconstruction cases for each material property, it is seen that the “GM+TV” loss case with the denoised slices as the target images yields the most accurate reconstructions.

7 Discussion

Results in the previous sections have shown the performance of the proposed reconstruction approach to be reasonably accurate and efficient. One further advantage of the proposed approach is that the reconstructed image is not constrained to be of the same size as the original image. Obtaining a high resolution image of a microstructure can be expensive, which limits the size and/or resolution of the obtained image. It is often of interest to analyze a microstructural image of dimensions larger than that of the original image. One of the useful features of this approach is the ability to reconstruct microstructure of spatial dimensions different from that of the original microstructural image. Here, as a demonstration, reconstruction of an image of size $700$ pixels $\times$ $1000$ pixels is considered using the “GM+TV+2PP” loss. The larger reconstructed image is shown in figure 19 along with the original slice $\#1$ microstructure for a visual comparison. To demonstrate the statistical equivalence of the larger reconstruction microstructure with its target image, the two-point probability function plot for the porous phase and the pore size complementary cdf plot for the two images are compared and shown in figure 20.

Another advantage of the proposed approach is that it is more efficient and accurate than other existing approaches based on statistical features of the microstructure. The approach is thus compared here with a modified version of the YT method [9, 10, 3] in terms of efficiency and accuracy. The YT method initiates by generating an image having the same porosity measure as the target microstructure. It is an iterative method where in each iteration, a randomly chosen black solid phase pixel and a white porous phase pixel are interchanged in the image. If the pixel swapping leads to an improvement in the statistical measure (equivalent to a reduction in loss function measure) of the new image compared to the previous one, the new image is updated to be the current reconstructed image. Otherwise, the previous image remains the current reconstructed image at the end of that iteration. The pixel swapping is also allowed if a random number generator, generating random numbers uniformly between $0$ and $1$ , returns a value greater than or equal to $p_{I}$ . Here $p_{I}$ is defined as the pixel interchange probability. The iterative convergence of the reconstructed image to the target image is measured by the root of sum of squares error (RSSE) between the two-point correlation function of the reconstructed image and the target image. With increase in the number of iterations, RSSE has a decreasing trend and the reconstructed image converges closer to the target microstructure with respect to the two-point correlation function measure.

Figure 21 shows a visual comparison of the reconstructed images formed by using the proposed reconstruction approach and the YT method. Slice $\#1$ , shown in figure 21a, is used as the target image in this case. It is clearly seen that the quality of the reconstructed image obtained by the proposed approach is better than that of the YT method. In this case, the YT method is run for $1$ million iterations and it takes around $90$ minutes to obtain the image in figure 21c. On the other hand, the proposed algorithm takes $17$ , $400$ iterations and only around $18$ minutes to produce the image in figure 21b. Thus, the proposed approach is found to be much more efficient and accurate than the YT method in the reconstruction of the porous ceramic material. It is noted that the algorithms are run on Google Colaboratory Pro [54], a cloud-based Jupyter notebook environment.

8 Conclusions

In this paper, a modified version of an existing transfer learning reconstruction approach [33] is presented by introducing additional loss functions in the optimization framework. It is then used to reconstruct a binary porous ceramic material. A thorough statistical study was done to check the robustness of the algorithm over different initializations and over different microstructure slices (original and denoised) of the same material. Furthermore, the reconstruction quality is assessed not only based on the statistical properties of the microstructure but also the FEA simulated effective material properties. This algorithm has the advantage of being easily extended to 3D microstructure reconstruction [55, 56, 57] if a suitable pretrained 3D convolutional network is available. It also has the flexibility of generating microstructures of size different from that of the target microstructure. A potential future work can be incorporating the simulated effective material property information within the reconstruction algorithm framework in an efficient manner, in order to generate microstructures with effective material properties closer to that of the target microstructure.

Acknowledgements

Research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-12-2-0023 and W911NF-12-2-0022. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

References

[1] R. Hazlett, Statistical characterization and stochastic modeling of pore networks in relation to fluid flow, Mathematical Geology 29 (6) (1997) 801–822.
[2] M. D. Rintoul, S. Torquato, Reconstruction of the structure of dispersions, Journal of Colloid and Interface Science 186 (2) (1997) 467–476.
[3] L. Graham-Brady, X. F. Xu, Stochastic morphological modeling of random multiphase materials, Journal of Applied Mechanics 75 (6) (2008).
[4] B. Collins, K. Matous, D. Rypl, Three-dimensional reconstruction of statistically optimal unit cells of multimodal particulate composites, International Journal for Multiscale Computational Engineering 8 (5) (2010).
[5] M. Talukdar, O. Torsaeter, M. Ioannidis, J. Howard, Stochastic reconstruction of chalk from 2d images, Transport in Porous Media 48 (1) (2002) 101–123.
[6] K. Gerke, M. Karsanina, E. Skvortsova, Description and reconstruction of the soil pore space using correlation functions, Eurasian Soil Science 45 (9) (2012) 861–872.
[7] Y. Jiao, F. Stillinger, S. Torquato, Modeling heterogeneous materials via two-point correlation functions. II. Algorithmic details and applications, Physical Review E 77 (3) (2008) 031135.
[8] T. Tang, Q. Teng, X. He, D. Luo, A pixel selection rule based on the number of different-phase neighbours for the simulated annealing reconstruction of sandstone microstructure, Journal of Microscopy 234 (3) (2009) 262–268.
[9] C. Yeong, S. Torquato, Reconstructing random media, Physical Review E 57 (1) (1998) 495.
[10] S. Torquato, C. Yeong, Reconstructing random media. II. Three-dimensional media from two-dimensional cuts, Physical Review E 58 (1) (1998) 224–233.
[11] C. M. Breneman, L. C. Brinson, L. S. Schadler, B. Natarajan, M. Krein, K. Wu, L. Morkowchuk, Y. Li, H. Deng, H. Xu, Stalking the materials genome: A data-driven approach to the virtual design of nanostructured p olymers, Advanced Functional Materials 23 (46) (2013) 5746–5752.
[12] H. Xu, Y. Li, C. Brinson, W. Chen, A descriptor-based design methodology for developing heterogeneous microstructural materials system, Journal of Mechanical Design 136 (5) (2014).
[13] H. Xu, D. A. Dikin, C. Burkhart, W. Chen, Descriptor-based methodology for statistical characterization and 3D reconstruction of microstructural materials, Computational Materials Science 85 (2014) 206–216.
[14] J. A. Quiblier, A new three-dimensional modeling technique for studying porous media, Journal of Colloid and Interface Science 98 (1) (1984) 84–102.
[15] P. Levitz, Off-lattice reconstruction of porous media: critical evaluation, geometrical confinement and molecular transport, Advances in Colloid and Interface Science 76 (1998) 71–106.
[16] J. W. Cahn, Phase separation by spinodal decomposition in isotropic systems, The Journal of Chemical Physics 42 (1) (1965) 93–99.
[17] S. Yu, Y. Zhang, C. Wang, W.-k. Lee, B. Dong, T. W. Odom, C. Sun, W. Chen, Characterization and design of functional quasi-random nanostructured materials using spectral density function, Journal of Mechanical Design 139 (7) (2017).
[18] J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks 61 (2015) 85–117.
[19] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444.
[20] N. Cristianini, J. Shawe-Taylor, et al., An introduction to support vector machines and other kernel-based learning methods, Cambridge University Press, 2000.
[21] C. K. Williams, Prediction with gaussian processes: From linear regression to linear prediction and beyond, in: Learning in Graphical Models, Springer, 1998, pp. 599–621.
[22] A. Bhaduri, L. Graham-Brady, An efficient adaptive sparse grid collocation method through derivative estimation, Probabilistic Engineering Mechanics 51 (2018) 11–22.
[23] A. Bhaduri, Y. He, M. D. Shields, L. Graham-Brady, R. M. Kirby, Stochastic collocation approach with adaptive mesh refinement for parametric uncertainty analysis, Journal of Computational Physics 371 (2018) 732–750.
[24] A. Bhaduri, J. Gardner, C. F. Abrams, L. Graham-Brady, Free energy calculation using space filled design and weighted reconstruction: a modified single sweep approach, Molecular Simulation 46 (3) (2020) 193–206.
[25] A. Bhaduri, D. Brandyberry, M. D. Shields, P. Geubelle, L. Graham-Brady, On the usefulness of gradient information in surrogate modeling: Application to uncertainty propagation in composite material models, Probabilistic Engineering Mechanics 60 (2020) 103024.
[26] A. Bhaduri, C. S. Meyer, J. W. Gillespie Jr, B. Z. Haque, M. D. Shields, L. Graham-Brady, Probabilistic modeling of discrete structural response with application to composite plate penetration models, arXiv preprint arXiv:2011.11780 (2020).
[27] B. L. DeCost, T. Francis, E. A. Holm, Exploring the microstructure manifold: image texture representations applied to ultrahigh carbon steel microstructures, Acta Materialia 133 (2017) 30–40.
[28] N. Lubbers, T. Lookman, K. Barros, Inferring low-dimensional microstructure representations using convolutional neural networks, Physical Review E 96 (5) (2017) 052111.
[29] R. Cang, Y. Xu, S. Chen, Y. Liu, Y. Jiao, M. Yi Ren, Microstructure representation and reconstruction of heterogeneous materials via deep belief network for computational material design, Journal of Mechanical Design 139 (7) (2017).
[30] H. Lee, R. Grosse, R. Ranganath, A. Y. Ng, Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations, in: Proceedings of the 26th Annual International Conference on Machine Learning, 2009, pp. 609–616.
[31] X. Li, Z. Yang, L. C. Brinson, A. Choudhary, A. Agrawal, W. Chen, A deep adversarial learning methodology for designing microstructural material systems, in: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Vol. 51760, American Society of Mechanical Engineers, 2018, p. V02BT03A008.
[32] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: Advances in Neural Information Processing Systems, 2014, pp. 2672–2680.
[33] X. Li, Y. Zhang, H. Zhao, C. Burkhart, L. C. Brinson, W. Chen, A transfer learning approach for microstructure reconstruction and structure-property predictions, Scientific Reports 8 (1) (2018) 1–13.
[34] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014).
[35] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009, pp. 248–255.
[36] S. Nickerson, Y. Shu, D. Zhong, C. Könke, A. Tandia, Permeability of porous ceramics by X-ray CT image analysis, Acta Materialia 172 (2019) 121–130.
[37] P. Kočí, M. Isoz, M. Plachá, A. Arvajová, M. Václavík, M. Svoboda, E. Price, V. Novák, D. Thompsett, 3D reconstruction and pore-scale modeling of coated catalytic filters for automotive exhaust gas aftertreatment, Catalysis Today 320 (2019) 165–174.
[38] S. Torquato, H. Haslach Jr, Random heterogeneous materials: microstructure and macroscopic properties, Applied Mechanics Reviews 55 (4) (2002) B62–B63.
[39] E. Vanmarcke, Random fields, 1983.
[40] S. Prager, Interphase transfer in stationary two-phase media, Chemical Engineering Science 18 (4) (1963) 227–231.
[41] H. H. Aghdam, E. J. Heravi, Guide to convolutional neural networks, New York, NY: Springer 10 (2017) 978–973.
[42] R. Venkatesan, B. Li, Convolutional neural networks in visual computing: a concise guide, CRC Press, 2017.
[43] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Communications of the ACM 60 (6) (2017) 84–90.
[44] V. Romanuke, Appropriate number and allocation of relus in convolutional neural networks, Scientific News of the National Technical University of Ukraine, Kyiv Polytechnic Institute (1) (2017) 69–78.
[45] D. Ciregan, U. Meier, J. Schmidhuber, Multi-column deep neural networks for image classification, in: 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2012, pp. 3642–3649.
[46] K. Yamaguchi, K. Sakamoto, T. Akabane, Y. Fujimoto, A neural network for speaker-independent isolated word recognition, in: First International Conference on Spoken Language Processing, 1990.
[47] L. Gatys, A. S. Ecker, M. Bethge, Texture synthesis using convolutional neural networks, in: Advances in Neural Information Processing Systems, 2015, pp. 262–270.
[48] L. I. Rudin, S. Osher, E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D: Nonlinear Phenomena 60 (1-4) (1992) 259–268.
[49] Hibbett, Karlsson, Sorensen, ABAQUS/Standard: User’s Manual, Vol. 1, Hibbitt, Karlsson & Sorensen, 1998.
[50] A. S. Dalaq, D. W. Abueidda, R. K. A. Al-Rub, I. M. Jasiuk, Finite element prediction of effective elastic properties of interpenetrating phase composites with architectured 3D sheet reinforcements, International Journal of Solids and Structures 83 (2016) 169–182.
[51] K. Ramani, A. Vaidyanathan, Finite element analysis of effective thermal conductivity of filled polymeric composites, Journal of Composite Materials 29 (13) (1995) 1725–1740.
[52] T. L. Bergman, F. P. Incropera, D. P. DeWitt, A. S. Lavine, Fundamentals of heat and mass transfer, John Wiley & Sons, 2011.
[53] S. Whitaker, Flow in porous media I: A theoretical derivation of Darcy’s law, Transport in Porous Media 1 (1) (1986) 3–25.
[54] E. Bisong, Google Colaboratory, in: Building Machine Learning and Deep Learning Models on Google Cloud Platform, Springer, 2019, pp. 59–64.
[55] V. Sundararaghavan, N. Zabaras, Classification and reconstruction of three-dimensional microstructures using support vector machines, Computational Materials Science 32 (2) (2005) 223–239.
[56] B. Bochenek, R. Pyrz, Reconstruction of random microstructures—-a stochastic optimization problem, Computational Materials Science 31 (1-2) (2004) 93–112.
[57] Y. Bhandari, S. Sarkar, M. Groeber, M. Uchic, D. Dimiduk, S. Ghosh, 3d polycrystalline microstructure reconstruction from fib generated serial sections for fe analysis, Computational Materials Science 41 (2) (2007) 222–235.