Hyperspectral imaging for dynamic thin film interferometry

The theory of thin film interference was formalized in the early 19^th century by Fresnel, and has since been discussed by many researchers in the context of measuring the thickness of thin films such as for bubbles [14], tear films [29], and for surface profiling [13]. Here we will briefly develop a formulation relevant for a hyperspectral camera.

Consider a beam of light having intensity $I_{0}(\lambda)$ incident on a thin liquid film of thickness $d$ and refractive index $n_{2}$. The film is bounded on top and bottom by media having refractive indices $n_{1}$ and $n_{3}$ respectively. Assuming normal incidence and non dispersive films, the reflected light intensity $I(d,\lambda)$ emanating from the thin film can be written as,

Here $\lambda$ is the wavelength of light, while $R_{1}$ and $R_{2}$ are the reflectivity coefficients obtained from the Fresnel equations evaluated for normal incidence, and are given by,

Finally, the intensity perceived by the $i^{th}$ channel of a pixel $H$ in a hyperspectral camera as a function of the film thickness can be computed as,

Here $I_{r}(\lambda)$ is the spectral response of filters in the system, and $S_{i}(\lambda)$ is the spectral sensitivity of the $i^{th}$ channel of a pixel.

During an experiment (Fig.1), a hyperspectral camera having $h$ channels at every pixel will encode reflections from a thin film of thickness $d$ as a $h$ dimensional vector. Utilizing Eq.4, we can invert this $h$ dimensional vector to recover the thickness of the thin film. In section 3, we will show that using a hyperspectral camera for this process instead of an RGB camera has significant advantages such as increased robustness against noise and absolute light intensity independent thickness determination.

The single bubble coalescence experiments used to validate the utility of hyperspectral imaging for thin film thickness measurements were performed using a modified Dynamic Fluid-Film Interferometer (DFI). The construction [11] and the utility [30, 31, 32] of the DFI has been previously discussed in a number of publications and in references therein.

For the current study, the DFI was modified to have a 16 channel snapshot HyperSpectral Imaging (HSI) camera (Model: MQ022HG-IM-SM4X4-VIS, Manufacturer: Ximea GmbH, Germany) as its top camera (Fig.1a). As the filter array inside the HSI camera has narrow spectral response bands (Supplementary Figure S4), the dichroic triband filter utilized with the top light for the same purpose (reducing the FWHM of spectral bands of a RGB camera)[11] was removed. The removal of the dichroic filter thus resulted in 120% increase in the luminous flux entering camera - improving the signal to noise ratio in the acquired hyperspectral interferograms.

To benchmark the thin film measurement capability of the hyperspectral camera, single bubble experiments were also performed using RGB cameras (IDS UI 3060CP), commonly used for thin film interferometery [11, 30].

To recover the film thickness from the hyperspectral image, the following steps were executed utilizing Matlab. Initially, the raw images from the snapshot HSI camera were sliced and spliced appropriately to reconstruct the hyperspectral cubes. Subsequently, background subtraction was performed on the cubes, followed by cropping, intensity correction and normalization. A k-Nearest neighbour search utilizing the cosine distance metric is performed between each pixel in the resulting HSI cube and the theoretical spectral map (Fig.2) generated from Eq.4. The thicknesses obtained as the first nearest neighbour in the k-Nearest neighbour search is used to construct the initial estimate of the thickness profile.

Finally, a spatial optimization algorithm is utilized to correct for the any incorrectly assigned points. The algorithm basically enforces the $C^{0}$ spatial continuity in film thickness by replacing any incorrectly assigned thickness with an appropriate thickness from the k possible thickness values at that point.

The performance of the new camera system is established by reconstructing the dynamic film thickness of a bubble in a silicone oil mixture. The results before and after optimization are compared to the manually reconstructed thickness profiles [11] in Fig.3. The reconstructed thickness profile before optimization (Fig.3.c) broadly resembles the manually reconstructed profile (Fig.3.b). However, due to noise and spectral mixing in the interference data, there are regions in the reconstructed profile that have physically unrealistic gradients. These artifacts are removed utilizing the optimization routines, and the resultant thickness profile after optimization (Fig.3.d) is seen be very similar to the manually reconstructed profile. The mean film thickness ($\iint T(x,y)dxdy/\iint dxdy$), a very common metric used to report the thickness of thin films [11, 30, 33], is very similar across the three cases with values of $358.9\;nm$, $344.4\;nm$ and $347.9\;nm$ for the manual, unoptimized and optimized thickness profiles respectively.

The absolute pixel-wise height difference between the manually reconstructed and the automatically reconstructed thickness profiles pre and post optimization are shown in Fig.3.e,f. Clearly, the reconstructions are very accurate with over $80\%$ of the pixels differing by less than $100\;nm$ for the unoptimized case, while over $90\%$ of the pixels differ by less than $100\;nm$ for the optimized case. The remaining $10\%$ of the regions have errors larger than $100\;nm$, and are most likely a result of inadequate background subtraction as well as due to the lac of pixel by pixel calibration information for the camera. As a reference, its also worth noting that reconstructions using the 3 channel data available from a RGB camera has an unacceptable accuracy, with less than $20\%$ of the pixels being classified within $100\;nm$ (Supplementary Figure S3).

Finally, a convenient advantage of the automated reconstruction is the ease of analyzing time sequential data to obtain the temporal evolution of the film thicknesses. Supplementary Video V1 shows one such reconstruction of a temporally evolving film thickness profile alongside its corresponding raw interferogram.

Theoretically, the 3 channels in RGB interferograms are sufficient to disambiguate the underlying film thicknesses [25]. However in practice, the inherent noise in the acquired image data breaks this theoretical uniqueness between the RGB intensities and the film thickness; thus requiring optimization (even for static films) [34] and/or calibration [25] for thickness recovery. Unlike the RGB interferograms, hyperspectral interferograms are relatively more robust to noise.

To illustrate the robustness of hyperspectral interferograms against noise, we construct a ramp thickness profile having a minimum thickness of $1\;nm$ and a maximum thickness of $2000\;nm$ (Supplementary Figure S1). This thickness profile is mapped to corresponding color interferograms using the color maps of the tested cameras. Subsequently, Gaussian noise (see Supplementary Text is added to the images. Inset in fig.4(a) shows one such RGB image before and after addition of the noise. Reconstruction algorithms (without optimization) are then used to predict the thickness ($T$) at each pixel in the noisy images. $T$ can be compared to the ground truth thickness ($T^{g}$) to quantify the reconstruction accuracy. We visualize the reconstruction accuracy in fig.4(a) by plotting the number of pixels having an error of less than $50\mu m$ $\left(\sum_{i}\mathds{1}(|T^{g}_{i}-T_{i}|<50)\right)$ as a function of the number of channels in the camera.

From fig.4(a) we see that for an RGB camera only about 20% of the pixels are classified within acceptable error. However, for a HSI camera (utilizing all 16 channels) we are able to recover the thickness from 97% of the pixels within acceptable error; thus showing that HSI cameras are more robust to noise. For the tested HSI camera, we also obtain similar reconstruction accuracy using either the real or virtual filters (Supplementary Figure S5) on a HSI camera, thus suggesting any type of filter may be used in the reconstruction process. Further we also see that having an ideal light (uniform intensity across wavelengths) or ideal filters (responses given by hat functions) can enhance the reconstruction performance; with ideal filters having the greatest impact on reconstruction accuracy (see Supplementary Text). Finally, its also interesting to note that the results indicate that a 3 channel camera (RGB camera) with perfect filters has a performance comparable to a regular HSI camera utilizing all 16 channels.

The increased robustness of HSI interferograms against noise is related to the higher dimensionality of the color co-ordinates that correspond to a given film thickness. As a result of the higher dimensionality, the pair wise Eucledian distances of HSI color co-ordinates are larger than the corresponding pair wise distances of color co-ordinates obtained from a RGB camera. This observation is quantified in fig.4(b) by plotting the cumulative pair wise distance distribution of the color co-ordinates (resolved at every nanometer) for a HSI and a RGB camera for two different film thickness ranges - $[0,2000]$ and $[0,4000]\;nm$. From fig.4(b), we clearly see that for a given thickness range, the curves corresponding to HSI cameras as compared to RGB cameras are shifted towards larger distances ($r$). Hence, as a consequence of the increased separation of HSI color co-ordinates, perturbations due to noise are less likely to result in a color co-ordinate becoming similar to another.

Another advantage of using hyperspectral imaging is that the absolute intensity of light at any point in the interferogram may be neglected during thickness reconstruction. This can shown be true using two different arguments. Firstly, standard transformations (such as RGB to HSV) can isolate the intensity information into a single channel. Neglecting the intensity channel, still gives us (for the tested HSI camera) information from 15 channels to unambiguously (see Fig.4) reconstruct the thickness. Since an RGB camera has only 3 channels, neglecting the intensity makes it impossible to unambiguously determine the thickness. Secondly, the cosine distance metric utilized in this paper to determine the film thicknesses completely ignores the intensity information. Despite ignoring the intensity information, we are able to recover the thickness profiles quite well (Fig.3.d), thus confirming that hyperspectral imaging enables thickness reconstruction without necessarily requiring information about the absolute light intensity.

A direct consequence of this result is that the reconstruction techniques described in this paper are robust against the natural spatial variation of incident light intensity (vignetting) over the interferogram. Hence in addition to obviating the need for accurately obtaining the absolute light intensities, the corrections for vignetting may also be conveniently avoided. Note that flat field corrections may still be required if there are spatial variations in pixel sensitivities in a camera.

Hyperspectral imaging when applied to thin film interferometry has some interesting characteristics. Firstly, the number of spectral classes is higher than in traditional hyperspectral imaging used for remote sensing or medical imaging. The number of spectral classes in thin film interferometry goes as $\mathcal{O}(T_{max}/T_{resolution})$, which is equal to $4000$ when trying to resolve film thickness of upto $T_{max}=4000\;nm$ with a resolution of $T_{resolution}=1\;nm$. Consequently, spectral matching routines used for thin film interferometry should have a high degree of specificity, and may also need optimization algorithms (as used in this paper) to completely reconstruct the thickness profile.

Secondly, the spectra vary gradually across spatially adjacent spectral classes. Unlike traditional hyperspectral imaging where there are no restrictions on spatially adjacent classes (and hence on the spectra), the spatial continuity of thin films restricts the spatially adjacent classes to correspond to contiguous film thicknesses. Hence, as a consequence of Fig.2, there is a gradual variation of the spectrum across spatially adjacent classes. Consequently, spectral mixing (due to lack of sufficient camera resolution) does not pose difficulties during thickness reconstruction, thereby obviating any need for spectral unmixing techniques [35].

Thirdly, as thin liquid films are dynamic, the spectral signatures change rapidly both in space and time. As a consequence, snapshot hyperspectral imaging (as opposed to techniques such as the pushbroom) is better suited for thin film interferometry.