A Comparative Study of Machine Learning Models Predicting Energetics of Interacting Defects

Defects in materials profoundly influence their properties. Understanding defects interactions is essential for optimizing material properties. Phenomena arising from defect interactions can occur from atomic-level to mesoscale and modeling across the scales is computationally challenging. Recently, machine learning potentials have shown success in accelerating materials simulations for molecules and crystals[Noé et al., 2020]. However, surface systems with a number of defects are rarely studied. Except defects interactions are complex, simulations of surface systems are costly since calculation for large supercells are needed [Sun and Ceder, 2013]. These prevent the progress of studies on complex phenomena in surface systems.
To explore the potential of applying machine learning techniques to study interacting defects on surfaces at atomic level, we provided as case study of single-type defects on pure element crystal. Driven from possible applications in real-life problem related to lithium battery[Liu and Lu, 2017], we focus on different numbers of vacancies on (100) surface of lithium (Figure 1). We defined the defect concentration on surface as

$$\textrm{defect concentration}=\frac{\textrm{number of vacancies}}{\textrm{sites of perfect crystal on surface}}\times 100\%,$$

(1)

The prediction task is, given the structure of defects distribute on surface to predict the free energy change $\Delta G$ when vacancies present on the surface. This free energy change is defined as

$$\Delta G=E_{\textrm{vac}}+nE_{\textrm{Li}}-E_{\textrm{Li slab}},$$

(2)

where $E_{\textrm{vac}}$ is the ground state energy of lithium surface with defects from Density Functional Theory calculation(DFT), $n$ is the number of vacancies in this system and $E_{\textrm{Li}}$ is the chemical potential of lithium (atomic energy of lithium in its most stable form), and $E_{\textrm{Li slab}}$ is the ground state energy of a pristine lithium supercell.
Our initial data set calculated by DFT has 88 structures with concentration from 6.25% to 50%. We split the training set and test set by concentration to make the training set and test set dissimilar both in configurations and concentrations. The training set contains structures with concentration less than 40%, and there are 73 structures. The test set contains the rest 15 structures with concentration higher than 40%. We first study the model performance on initial test data with three possible approaches. Cluster expansion(CE) is the the approach we showed it can predict the target at high accuracy. We further generate more surface systems with different defect configurations and their energy using this cluster expansion model trained with initial data, and use these configuration-energy pairs to test the potential of applying graph neural network(GNN) in this type of systems. Contributions of this work include (1) Qualitative reveal the limitation of machine learning potentials for systems with interacting defects and provide a model to alleviate; (2) Establish a computationally feasible way to test the potential of GNN for surface with intereacting defects.

The models were trained on systems with low defect concentrations and then tested on systems with high defect concentrations. We explored 3 kinds approaches: (1) descriptors + ML algorithms, (2) graph neural networks, (3) cluster expansion. For the first two approaches, we presented the prediction statistics of the most representative models, they are (1) MBTR[Huo and Rupp, 2022] with linear regression, (2) DimeNetGasteiger et al. [2022]. We tested the best number of atoms to include in representation and the results in mean absolute error(MAE) and coefficient of determination($R^{2}$) are in Table 1 and Figure 2. All the models training with representations building from 2 layers of atoms, which are surface and subsurface atoms, have the lowest test MAE. Cluster expansion model can predict the energy with high concentration defects most accurately and achieving negligible error compare to DFT level accuracy. Linear regression model with MBTR also has low test MAE, however, the prediction error is higher when the concentration of defect is higher. DimeNet has the worst performance compare to the last two models, this is due to the training data is limited. Since the most important factor that impact the energy of this kind of extended system is surface configuration, when building a prediction model, information about the atoms far from surface makes less contribution in determining the free energy change of the system. This match well with the results in Figure 1 comparing column-wise. Notably, in graph neural network, if we represented the supercell structures as graphs while only surfaces differ, the model will treat every structure the same.

Since cluster expansion model is superior in prediction accuracy and uncertainty, we use this cluster expansion model to generate more target energies with different surface configurations at low concentrations and support the training of DimeNet. Using the energies predicted by cluster expansion as surrogates, it bypass the need of computational cost for thousands of supercell of DFT calculationsto test the potential of DimeNet applied at systems with interacting defects. We trained DimeNet with 500 and 5,000 CE generated energies respectively and the results are in Figure 3 and Table 2. As there are more training data points, DimeNet gains more expressive representation. When training with 5,000 configuration, DimeNet is able to predict as good as MBTR designed by domain expert. This finding motivate us to do further studies on how DimeNet captures relationship between defects and interactions by utilizing more low concentration defect configurations and whether the representation generalize to high concentration systems.

In this work, we explored three approaches to predict the free energy change of surface systems with interacting defects with a small data set. While materials descriptor with classical machine learning algorithm performs well with limited data, cluster expansion is more accurate and less biased in prediction. Within the scope of interacting defects scattered on surface, cluster expansion model can serve as a means to provide training data at low cost to survey the potential of applying deep neural networks in the task of property prediction for surface with interacting defects. On the other hand, it is worth considering the necessity of using machine learning with descriptors or deep learning to study the defects interactions at surface only when cluster expansion model performs well with purchasable cost to obtain training data.

This project was conducted independently during my early stage of graduate studies and may not have received formal supervision. If you come across this report, please feel free to contact me with any issues or suggestions you may have.