Deep Spectral CNN for Laser Induced Breakdown Spectroscopy

The pre-processing task aims at disentangling the sources of sensor uncertainty and measurement effects from the spectral signal itself. Such sources include the plausible effects of dark background currents, white noise, electron continuum, wavelength offsets, instrument response and target-to-detector range corrections [1]. We begin modeling the effects of dark current background, white noise and electron continuum in LIBS signals using the additive noise model

Here, $\mathbf{y}\in\mathbb{R}^{N}$ is the the raw measured LIBS signal, $\mathbf{x}\in\mathbb{R}^{N}$ is the clean signal and $\mathbf{z}\in\mathbb{R}^{N}$ is the noise including dark current, ‘white’ noise and background continuum. Although not explicit; such noise model is also used in [4] and can be derived through the sequential application of dark current subtraction, white noise denoising and background continuum removal methods therein.

The problem of denoising or cleaning in Eq. (1) is tackled here by learning a function that discriminates the LIBS signal $\mathbf{x}$ from the sensor noise $\mathbf{z}$. In some sense, such formulation assumes the distribution of sensor noise has a structure given the measurements $\mathbf{y}$ that is exploited by learning. We propose to do this in a residual learning framework through a deep spectral convolutional neural network (CNN) that predicts a residual function $\mathcal{R}:\mathbb{R}^{N}\times\mathbb{R}^{L_{1}}\rightarrow\mathbb{R}^{N}$ satisfying:

where comparing with Eq. (1) implies $\hat{\mathbf{z}}=\mathcal{R}(\mathbf{y},\boldsymbol{\Theta}_{1})$. Such residual framework (as opposed to directly estimating the clean signal $\mathbf{x}$) gives in a DL context the flexibility to use high performance networks with lower approximation error and faster training rate capabilities [12]. Learning weights $\boldsymbol{\Theta}_{1}\in\mathbb{R}^{L_{1}}$ is performed through the optimization of a loss function using a training set of $M$ raw-clean (level1a) signal example pairs $\{(\mathbf{y}_{m},\mathbf{x}_{m})\}_{m=1}^{M}$. This loss function $f_{1}:\mathbb{R}^{L_{1}}\rightarrow\mathbb{R}$ is the average $\ell_{2}$-norm error between ground truth and estimation residuals described by:

Once the weights $\boldsymbol{\Theta}_{1}$ are learned, the spectral CNN gradually removes the clean signal $\mathbf{x}$ from the input signal $\mathbf{y}$ as it goes through the layers of the network in feed-forward mode.

The spectral CNN architecture used is shown in Figure 1 and composed of Module-1 and Module-2. Module 1 consists of a convolution and rectifier linear unit (ReLu) layer followed by a sequence of $D$ convolution, batch normalization (Bn) and ReLu interleaved layers while Module-2 is a convolution and subtraction of the residual estimation. A similar architecture than the one used for pre-processing here was proposed in the context of image denoising in [13] and resulted in state of the art performance over sophisticated handcrafted methods such as the BM3D [14] and denoising by soft-thresholding in a wavelet decomposition [15].

Additional pre-processings of the LIBS signal such as compensation for the modulation induced by the instrument response function (IRF) is also addressed here in a residual based formulation. The model, loss function and architecture is the same as those described in Eq. (2) and (3) and Figure 1, respectively. However, the difference lies in the training stage of the spectral CNN where instead of training with raw-clean(level1a) signal pairs we train with $M$ raw-clean (level1b) signal example pairs $\{(\mathbf{y}_{m},\mathbf{x}_{m})\}_{m=1}^{M}$. Note though, that we do not place attention into modifying the signal axis to be in the appropriate scale of photons per DN (intensity) as in [4]. Reason being that in a multivariate analysis approach and from an information theoretic standpoint scaling a LIBS signal does not bring any new information that can further improve signature analysis. We thus make our framework invariant to scaling and additional normalizations in [4] by normalizing raw and pre-processing signals to be of unit $\ell_{2}$-norm. As a final note to this subsection we would like to mention that our framework is flexible to include only the user desired pre-processings. For example, if the user wants to independently collect a number of dark current signals and subtract them separately this can be done by training the network with the appropriate pairs $\{(\mathbf{y}_{m},\mathbf{x}_{m})\}_{m=1}^{M}$ omitting dark current subtractions. With regards to wavelength adjustments, we omit this step from our framework as it is related to the association of wavelengths to the independent axis index of the LIBS signal and instead recommend usage of the method proposed in [4].

Further processing aiming at getting the chemical element constituents from the LIBS signal signatures is also treated here. This problem, also referred to as calibration in the LIBS context consists in the estimation of a vector $\mathbf{v}\in\mathbb{R}^{C}$ representing each entry a chemical compound and the value at each entry a quantitative measure or estimate of the relative to $\%$-oxide percentage. The number $C$ represents the total number of chemical elements to be determined and is user-defined. The problem is generalized thus here as finding an operator $\mathcal{F}:\mathbb{R}^{N}\rightarrow\mathbb{R}^{C}$ that takes as input a LIBS signal $\mathbf{x}\in\mathbb{R}^{N}$ and outputs the vector $\mathbf{v}$ of chemical composition/concentration as given in Eq. (4) as:

Similar to [6], we assume a linear regression calibration model and let $\mathcal{F}(\mathbf{x},\boldsymbol{\Theta}_{2})$ be learned by a shallow neural network with weights $\boldsymbol{\Theta}_{2}\in\mathbb{R}^{L_{2}}$. The neural net employed consists of a set of convolutional and fully connected (FC) layers connected as shown in Module-3 of Figure 1 which regress LIBS signatures to chemical content.

Learning the weights $\boldsymbol{\Theta}_{2}$ is done in training with example pairs $\{(\mathbf{x}_{m},\mathbf{v}_{m})\}_{m=1}^{M}$ of pre-processed LIBS signals and corresponding chemical constituents. The optimization minimizes the loss function $f_{2}:\mathbb{R}^{L_{2}}\rightarrow\mathbb{R}$ measuring the average $\ell_{2}$-norm error between a vector representing the ground truth element concentration labels $\mathbf{v}_{m}\in\mathbb{R}^{C}$ and the vector of estimates $\hat{\mathbf{v}}_{m}\in\mathbb{R}^{C}$ produced by the network. Such loss function can be represented as given by Eq. (5):

The input $\mathbf{x}_{m}$ in (5) is the pre-processed spectrum obtained in a feed-forward pass through the trained spectral CNN as in Section 2.1.1 or any other suitable method such as [4]. We refer to this type of training as training by composition as it is composed of two independent stages: (1) for pre-processing and (2) for calibration both trained independently and under supervision. The overall architecture pipeline is illustrated in Figure 1. One of the benefits of such training scheme is that it offers access to the intermediate pre-processed signal which can be useful and/or necessary for user interpretation. However, this may come with a cost in performance in that it is expected to have a pre-processing quality no better than that of the training pre-processed ground truth signals $\mathbf{x}_{m}$.

We also propose a training scheme were the network architecture in Figure 1 learns the meaningful LIBS signatures without supervision and in an end-to-end fashion. In such a scheme the network learns a function $\mathcal{F}^{\prime}:\mathbb{R}^{N}\times\mathbb{R}^{L_{3}}\rightarrow\mathbb{R}^{C}$ with weights $\boldsymbol{\Theta}_{3}\in\mathbb{R}^{L_{3}}$ that takes a raw LIBS signal $\mathbf{y}\in\mathbb{R}^{N}]$ as input and estimates its chemical constituents $\mathbf{v}\in\mathbb{R}^{C}$ as output. The spectral CNN is trained with backpropagation using raw LIBS and chemical constituent example pairs $\{(\mathbf{y}_{m},\mathbf{v}_{m})\}_{m=1}^{M}$. The optimization seeks to minimize the average $\ell_{2}$-norm error between ground truth and estimates as in Eq. (6)

The benefits of such end-to-end training scheme are two-fold: (1) it no longer requires supervision for pre-processing and instead lets the network learn the optimal solution in the sense of (6) and (2) we can achieve lower error as we do not introduce additional noise to the system through the estimation of $\mathbf{x}_{m}$ as in Eq. (5). In the next section, we validate the proposed approach empirically in both pre-processing and calibration tasks with data obtained by Chemcam in both Mars and earth lab environments.

The training stage aimed to learn raw (edr)-to level1a (rdr) pre-processings as described in Section 2.1.1 uses as the training set the raw and level1a corresponding pairs as input and label, respectively. Such level1a pre-processing includes background subtraction, white noise denoising and continuum removal. A representative example of a result and comparison against that of [4] in the test ’Mars’ dataset is included in Figure 2. Note that in general, the proposed method was able to learn and predict the pre-processing method of [4] well, without the prior information requirements (e.g., dark current acquisitions and estimates) while also coping well with the differences among all three detector types (i.e., UV, VIS, NIR). In general, most of the peaks obtained by the pre-processing in [4] where well approximated by the proposed deep spectral CNN. For reference, the $\ell_{2}$-norm error between [4] and the proposed method for the specific case shown in Figure 2 is 0.018 with a maximum unit $\ell_{2}$-norm error normalization.

Learning to pre-process at level1b which in addition to including the steps of level1a accounts for instrument response function (IRF) adjustments and other data normalizations to account for, for example, sensor-to-target distance was carried out as a task by the same approach described in 2.1.1. The process to learn such pre-processing consists on training the network with raw (edr) and level1b (ccs) corresponding pairs. Experiments were conducted on both ’Mars’ and ’Calib’ datasets, independently. Representative example results from the test ’Mars’ and ’Calib’ datasets are included in Figure 3 and Figure 4, respectively.

In both cases, the proposed spectral CNN is capable of learning and subtracting good approximations to the background dark spectrums of [4]. This with the additional advantage of removing the dark currents spectrum collection requirements prior to material sampling. In addition, as is illustrated specially in Figure 3.d, the proposed method seems to be better at reducing the overlap between neighboring peaks than [4] a matter indicative of a more effective electron continuum removal. The spectral CNN was also capable of learning and effectively adjusting for the IRF for all three detectors (UV, VIS, NIR) similar to the methodology in [4]. For reference, the $\ell_{2}$ difference between [4] and the proposed spectral CNN pre-processing is of 0.026 and 0.068 for Figure 3 and Figure 4, respectively, each with a max unit $\ell_{2}$-norm error normalization.

Table 1 summarizes the root mean squared error (RMSE) expectation overall examples in the test datasets in both ’Mars’ and ’Calib’ cases and at the two pre-processing levels experimented with, in this paper. Here, RMSE is measured by (3) with individual spectra normalized to unit $\ell_{2}$-norm.

Note that in all cases the proposed method shows relatively small RMSE values meaning the learned mappings are generally good approximations to the multi-step pre-processings done as in [4]. Note that in Table 1, Level 1a pre-processing results in the ’Calib’ dataset appears as ’N/A’ since labels were not available for such a case. In addition to these results, we also evaluate for any degradation in pre-processing performance as a function of detector-to-target distance. Table 2 includes the RMSE value results of the pre-processing performance as a function of source to target physical distance where distance is measured in meters (m). Note that such test includes only level 1b (ccs) pre-processings on the ’Mars’ dataset since the ’Calib’ dataset was obtained at a fixed distance throughout all standards.

Table 2 shows that the pre-processing RMSE performance of the proposed spectral CNN remains approximately constant across the different physical distances at which samples where queried. The interpretation of such result is that the spectral CNN is capable of learning the distance corrections applied by [4] and is thus equipped with some level of invariance. Such invariant capability appears to hold as long as a sufficient number of training examples per distance is available at the learning stage. Note that from 5-7m distances there seems to be a marginal RMSE loss in performance in the test partition. However, we attribute such loss to the estimation uncertainty from using a small sample size; in this case 310 and 88. We conclude the pre-processing experimentation by mentioning that in our implementations the spectral CNN was able to pre-process individual shots in a feed forward pass at rates $\sim$50 Hz on a CPU. Such rates render this method as computationally efficient with real-time deployment capabilities.

For learning by composition, we refer to the sequential pre-processing under supervision followed by the QQCC estimator also under supervision both trained independently. The two pre-processing methods compared to perform task (1) are: the method of [4] and the learned spectral CNN proposed here and trained as described in Section 3.1.2. For the QQCC estimation task (2), we used the parallel stack of linear regressor layers in Module-3 of Figure 1 on top of the result of each of the compared pre-processing methods. One independent linear regressor module is trained per compared pre-processing method using as input the result of pre-processing either by [4] or in a feed-forward pass through the proposed spectral CNN and as label the corresponding ground truth QQCC of the sample.

The linear regressor layers are composed of 8 parallel channels each operating to regress a single chemical element. Note that more parallel channels can be added or removed trivially depending on the number of major and/or minor chemical elements being estimated. For training the regressor layers, the ’Calib’ dataset was partitioned into randomly drawn $\sim 47,000$ and $\sim 31,000$ training and testing independent subset examples, respectively.

The experiments conducted to validate the proposed end-to-end learning scheme to estimate QQCC given a (single shot) raw spectral signal consists of measuring RMSE performance for each of the 8 major elements. Training was performed in an end-to-end fashion using pairs of (single shot) raw and corresponding major element concentrations in the ’Calib’ train set as input and label, respectively. Partition I used used a training set of $\sim 47,000$ randomly drawn (single-shot) examples while the test set is comprised of $\sim 31,000$ independent examples. Partition II instead splits the dataset into a training set of $\sim 54,000$ randomly drawn (single-shot) examples and a testing set comprised of $\sim 7,500$ drawn example shots ensuring that no shots coming from the same calibration target exists in both subsets.

A summary of the resulting predictions of QQCC for the test set of $\sim 31,000$ single shot raw (edr) inputs in partition I is shown in Figure 5. The horizontal axis corresponds to the ground truth certificate value whereas the vertical axis shows the prediction with the spectral CNN. The red line corresponds to the regression of the point estimates shown in black and unfilled circles while the black 1:1 line is the optimal regression. Note that in general most point estimates follow the 1:1 line in all eight chemical element cases, that the spread of points around the regressed lines is relatively small and that the regressed red lines are in general close to the 1:1 line. Chemical elements with larger regressed line deviation from the 1:1 line are those with no certificate value in larger oxide wt.$\%$ portions plausibly causing small miss-alignments at those regions. Such is the case for TiO₂ for which no high oxide wt. $\%$ is available in the reference calibration standards [6]. However, even in such case the resulting spread as measured by the RMSE summarized in the last column of Table 3 is significantly small.