Live processing of momentum-resolved STEM data for first moment imaging and ptychography

A continuous live scanning display for large-scale 4D-STEM data benefits greatly from a data processing method where smaller portions of input data are processed independently and merged progressively into the complete result. For virtual detectors and first moments, often referred to as "centre-of-mass (COM)", this is trivial to achieve since each detector frame, i.e., diffraction pattern, produces an independent entry in the final data set which allows frames to be processed individually, accumulating results in a buffer. Displaying the contents of this buffer at regular intervals provides a live-updating view.

In contrast, ptychography generates results by putting detector frames from the entire data set, or at least a local environment, in relation to each other. Consequently, adapting the methodology so as to circumvent the processing of the entire data set at once, thus gradually merging partial results extracted from portions of the input data into the complete result, is necessary. As previously demonstrated by Rodenburg & Bates (1992), Rodenburg et al. (1993) and Pennycook et al. (2015), both phase and amplitude information can be extracted from a set of diffraction patterns (usually restricting to the Ronchigram region) by performing the Fourier transform of the four-dimensional data set with respect to the scan raster and reordering the dimensions. Depending on the model presumed for the interaction between specimen and incident STEM probe, the direct inversion of the data can either be done by Wigner Distribution Deconvolution (WDD) or the SSB ptychography scheme (Rodenburg et al., 1993). Whereas WDD is based on a single interaction with an arbitrary complex object transmission function and is capable of separating specimen and probe, the weak phase approximation

governs the SSB approach. Here, the specimen exit wave at a given scan point can be expressed by a multiplication of the probe wave function $\psi_{\text{probe}}(\vec{r})$ with the first-order Taylor expansion of a phase object with phase distribution $\Phi(\vec{r})$. It is hence limited to weakly scattering ultra-thin objects e.g. thin light matter investigated with relativistic electrons. Any quantitative interpretation of SSB reconstructions of real data should, therefore, be examined critically since eq. (1) breaks down quickly with increasing thickness. Its capability of direct, dose-efficient phase recovery makes it nevertheless attractive for the in situ qualitative assessment of specimen and imaging conditions. A more advanced ptychography scheme can still be applied after recording. Because a data point in the final reconstruction depends on all recorded scan points, the reconstruction is only accurate if it is applied to the full 4D data.

We refer to original work for a derivation of the conventional SSB methodology (Rodenburg & Bates, 1992; Pennycook et al., 2015) and give a concise summary here. The first processing step consists of Fourier transforming the 4D data cube as to the scan coordinate, translating the scan coordinate in real space to spatial frequencies $\vec{Q}$ sampled by the scanning probe. This new 4D data cube can be ordered such that each scan spatial frequency defines one plane. By employing the weak phase object approximation in eq. (1), Rodenburg et al. have shown analytically that the data in each plane is described by three discs of the size of the probe-forming aperture, positioned at the origin and as Friedel pairs at the positions defined by the spatial frequency vector $\vec{Q}$. Importantly, double overlaps as depicted schematically in Fig. 1 contain the complex Fourier coefficients of $i\Phi$, potentially affected by aberrations of the probe-forming system. The double overlap regions are often referred to as trotters colloquially.

The ptychographic SSB reconstruction is a linear function of the input data since the result is obtained with a sequence of linear transformations, such as Fourier transforms, element-wise multiplication, and summation (Pennycook et al., 2015). Such linear functions are particularly suitable for incremental processing since they are additive. Mathematically, the complete input data can be understood as the sum of smaller individual data portions that are padded with zeros to fill the shape of the complete data set. Additive functions allow to calculate the complete result by accumulating the processing results of zero-padded portions in any subdivision and order. Furthermore, intermediate results can be extracted at any desired stage from the yet incomplete sum of results.

However, directly processing zero-padded data this way is very inefficient for computationally demanding algorithms such as ptychography employing large 4D-STEM data, because the processing effort and memory consumption is amplified by the number of subdivisions. For that reason, the algorithm should be reformulated to process smaller input data portions without zero-padding. The additivity and homogeneity of linear functions gives ample freedom to restructure the underlying data processing flow towards this goal, allowing development of mathematically equivalent implementations that are optimized for live imaging.

In the particular case of SSB ptychography, individual spatial frequencies of the result $\Phi(\vec{r})$ are extracted from spatial frequencies of signals at specific scattering angle ranges within the double overlaps (Rodenburg et al., 1993; Pennycook et al., 2015) introduced in Fig. 1.

Suppose we have the intensity of diffraction patterns present as four-dimensional data, which is written as $\mathbf{D}_{xy}\in\mathbb{R}^{m\times n}\quad\text{where}\quad x\in[s_{x}],y\in[s_{y}]$ and the notation $[s_{x}]$ represents the set of natural numbers not exceeding $s_{x}-1$. That is, $[s_{x}]:=\{0,1,\ldots,s_{x}-1\}$ with the total number of elements $s_{x}$. The indices $x,y$ represent the scan position index, with the total number of scanning steps being $s_{x}$ and $s_{y}$ in each direction. Each matrix $\mathbf{D}$ is a diffraction pattern $\left(d_{pq}\right)$ where $p$ and $q$ represent the pixel index on the detector with dimension $m\times n$. Each pixel index $pq$ corresponds to a scattering angle, or equivalently, spatial frequency in the specimen.

We can write the Fourier transform with respect to the scan raster as

where $k$ and $l$ denote the spatial frequencies in the scan dimension, taking the places of $x$ and $y$. Therefore we obtain a four-dimensional dataset in the spatial frequency domain, i.e., $\mathbf{F}_{kl}\in\mathbb{C}^{m\times n}$.

The next step in SSB is to apply a filter $\mathbf{B}_{kl}\in\mathbb{C}^{m\times n}$ for each tuple of spatial frequencies $kl$ that calculates the weighted average of the spatial frequency signal over specific ranges of pixels, i.e. scattering angles $pq$, that can have positive or negative weight as defined by the trotters. It should be noted that the structure of this filter is different for each tuple of spatial frequencies.

Using (2), we can write the reconstruction in the Fourier domain $p_{kl}$ as

The reformulation is given by interchanging the summation and the index dimension of detector and the index of scan points in the real and frequency domain, respectively. This nested summation can be pruned without changing the result by skipping parts that are known to yield zero, for example calculations for empty double overlap regions. Note that $p_{kl}$ essentially represents the Fourier coefficients of $i\Phi_{xy}$ in eq. (1) which are potentially affected by aberrations of the probe-forming system.

The inner part of eq. (3) can be implemented with a matrix product between $\mathbf{B}$ and $\mathbf{D}$. The outer sum over $x$ and $y$ can then be sub-divided and reordered to process the input data $\mathbf{D}$ incrementally in smaller portions. Numerically, the filter matrix $\mathbf{B}_{kl}$ can be stored efficiently as a sparse matrix since the trotters for the highest frequencies are often empty (no overlap), and for many frequencies the double overlap region where the filter is non-zero, is small.

As an intermediate summary, this formulation translates the SSB scheme to a matrix product of a partial input data matrix with a sparse matrix containing the double overlap regions. Elements of a Fourier transform are then applied to the result of this matrix product. This generates a partial reconstruction result in the frequency domain that covers the entire field of view. The partial reconstructions for all partial input data portions are accumulated in a global buffer using a sum, as described above. For a live view of the reconstruction in the spatial domain, the contents of this buffer can be inversely Fourier transformed at any desired time to show the transmission function of the specimen within the weak phase object approximation.

We used LiberTEM (Clausen et al., 2021) as a data processing framework since it is optimized for MapReduce-like approaches and designed with live data processing capabilities in mind (Clausen et al., 2020). LiberTEM user-defined functions (UDFs) provide the application programming interface (API) to efficiently implement operations that follow the described pattern: A method to process a stack of frames that is called repeatedly, task data to store constant data such as the sparse matrix for the double overlaps, result buffers of arbitrary type and shape, and user-defined merging operations to generate a result of arbitrary complexity from partial results. Furthermore, LiberTEM allows to update a result display each time a partial result is merged. UDFs for first moment analysis and virtual detectors were already implemented before for offline data analysis.

To run the UDFs on live data, we implemented a prototype live UDF back-end that allows to run a set of UDFs on data from a Quantum Detectors Merlin for EM Medipix3 for electron microscopy (Plackett et al., 2013; Quantum Detectors, 2019). It uses multiple CPU cores for decoding the raw data from the detector, and a single GPU for the main processing task, which was sufficient for this application. A production-ready version with support for multiple processing nodes, further enhanced use of multiple CPUs and multiple GPUs similar to the offline data processing capabilities of LiberTEM is being designed at the time of writing and will be published as open-source as a part of LiberTEM.

For comparison, ePIE (Maiden & Rodenburg, 2009) based ptychographic reconstructions have been performed in post processing as it can reconstruct both the complex object transmission function and the complex illumination, still presuming single interaction of probe and specimen, but considering an arbitrary complex object transmission function. This was done to check whether the straightforward SSB approach compromises the quality of the result compared to more advanced ptychographic schemes. It has to be noted that, as an iterative method, ePIE is not suitable for live imaging in the current realisations.

We demonstrate live processing using two different specimens. First, indium selenide (In₂Se₃) was used to highlight the advantages of live ptychography and centre of mass in comparison to conventional STEM (Ye et al., 1998). Second, a strontium titanate (SrTiO₃) lamella with the electron beam incident along the [100] axis for a more quantitative analysis was used. The latter is stable and provides good contrast in conventional STEM for comparison and adjustments, is well-characterized, and at the same time can highlight the ability to also resolve the light oxygen columns that are difficult to image with reliable contrast by conventional STEM techniques (Browning et al., 1995). Too small double overlap regions with an area of less than 10 px were omitted in live processing to avoid the introduction of excessive noise. This can be understood as a band-pass filter applied to exclude the high frequencies close to the transfer limit of SSB ptychography which corresponds to twice the radius of the probe-forming aperture.

First moment imaging and ptychography require the correct alignment and calibration of the scan (specimen) coordinate system with respect to the detector coordinate system. The convergence angle of the incident probe was measured with a polycrystalline gold specimen. Employing parallel illumination first, the (111) gold diffraction ring was used to calibrate the diffraction space assuming a lattice constant of gold of 0.4083 nm (Villars & Cenzual, 2016). With the known wavelength the convergence semi-angle was determined to 22.1 mrad from a Ronchigram recorded in the same STEM setting as used in the actual experiment.

The rotation of the detector coordinate system with respect to the scan axes was determined by minimizing the curl of the first moment vector field and making sure that the divergence of the field is negative at atom positions. Note that, in theory, the curl of purely electrostatic fields should vanish. The pixel size in the scan dimension was taken from the STEM control software during live processing and verified by comparison with the known lattice constant of SrTiO₃. The residual scan distortion, that is, the translation of the diffraction pattern as a whole during scanning, was not compensated for since it turned out to be negligible at the atomic-resolution STEM magnifications used in this analysis.

Data was acquired at a probe-corrected FEI Titan 80-300 STEM (Heggen et al., 2016) operated at $300\,$kV. The microscope was equipped with a Medipix Merlin for EM detector operated at an acquisition rate for a single diffraction pattern of $1$ kHz in continuous mode. The scan size was $128\times 128$ scan points and the recorded diffraction patterns had a dimension of $256\times 256$ pixel. In addition, the high-angle annular dark field (HAADF) signal has been recorded with a Fischione Model 3000 detector covering an angular range of 121.7 mrad to approximately 200 mrad. The upper limit is defined by apertures of the microscope rather than by the outer radius of the detector. The run time for each step of the data processing flow was measured using the line-profiler Python package.

The performance of our approach is demonstrated in Fig. 2 which depicts (a) the structure of In₂Se₃ together with the projected potential and (b) snapshots of the reconstructed phase in nanosheets of this material using SSB ptychography at different stages of the scan. The reconstruction is performed progressively such that only the diffraction patterns of incoming scan pixels are processed and added to the final reconstruction according to eq. (3) in which all pixels are affected. Whereas current research in the field of materials science explores ultra-thin In₂Se₃ as a candidate for 2D ferroelectrics, this specimen was chosen due to its robustness against electron dose for the methodological development. The data in Fig. 2 b was taken from a live video recorded during the STEM session, being available as supplementary material online.

In general, the quality of the ptychographic reconstruction strongly depends on matching the position and radius of the (usually circular) aperture function on the detector with the double-overlap regions. Even small deviations lead to a mismatch between the mask borders and the edges of the zero-order disk, which adds noise and reconstruction errors from those erroneous border pixels. A precise alignment can be done, e.g. using a position-averaged diffraction pattern or by recording the bright-field disc with the specimen removed from the field of view, as in our analyses.

It must be pointed out that a partial SSB reconstruction only approximates the result because not all spatial frequencies have been completely sampled at this point. However, it is already possible to see the atom columns in the partial reconstructions. This allows time- and dose-efficient live focusing and navigation to regions of interest on specimens. In Fig. 3, complete reconstructions of In₂Se₃ are shown. All atom columns can be seen clearly in the bright field images, in the divergence of the first moment and in the phase of the object transmission function obtained by SSB ptychography. All signals were calculated live from the 4D-STEM data stream.

Figure 4 depicts the phase of ptychographic SSB reconstructions of SrTiO₃ performed live while changing microscope parameters such as the STEM magnification (scan pixel size) in Fig. 4 a-c, and the probe focus in Fig. 4 d. The full video showing live updating during continuous scanning while microscope parameters are changed is available in the supplementary material online.

Although the inherent reconstruction parameters are naturally robust against a change of the probe focus for direct ptychography schemes that do not intend to correct aberrations, a magnification change influences the relation between a spatial frequency in the reconstruction in pixel coordinates, and the pixel distance in sample coordinates. This means that the geometry of a double overlap region that contains the signal to reconstruct a certain spatial frequency in the result changes with magnification. In the strict sense, changing the magnification without updating the scan step size (and double overlap masks) in the reconstruction is, therefore, mathematically inaccurate. In practice, however, it is desirable to be able to lower the magnification so as to navigate coarsely to a specimen region of interest without a new calibration or pre-calculation of the masks $\mathbf{B}_{kl}$ in eq. (3) to be time-efficient, and then observe or record details after switching back to the correct magnification again. Therefore, we studied to which extent the reconstruction is robust against a magnification change despite keeping the internal SSB parameters unchanged in Fig. 4 a-c. It showcases that the structural contrast is in general maintained. A possible future development with only a moderate effort could be pre-defining double overlap masks for common magnifications in order to build a look-up table that solves inaccuracies. We assume without loss of generality regarding the algorithmic structure, that live processing is required at a dedicated magnification being equal to the final recording where data is actually stored, and mention a certain robustness against changes of the scan pixel size as a valuable side note.

In our implementation the sparse matrix product was the throughput-limiting step. By using an efficient GPU implementation from cupyx.scipy.sparse (CuPy), this could be accelerated sufficiently to allow processing of over 1000 frames per second at suitable parameters, enabling for live imaging using a Medipix3 sensor.

The memory consumption of the current SSB ptychography implementation scales as $\mathcal{O}\left(N\right)$ with the number of scan points at constant aspect ratio since the number of spatial frequencies to reconstruct scales linearly with the number of scan points, and each reconstructed frequency adds a double overlap region. These regions sample the diffraction data more densely when extracting more spatial frequencies, meaning the average number of non-zero entries per trotter is roughly constant for a given pixel size and beam parameters. As the processing time for dense-sparse matrix products roughly scales linearly with the number of non-zero entries in the sparse matrix and number of vectors in the dense matrix, SSB scales poorly with $\mathcal{O}\left(N^{2}\right)$ in computation time with the number of scan points.

The size of the matrix containing the double overlap regions is highly dependent upon the acquisition parameters in the present implementation. A larger camera length increases the Ronchigram size, consequently increasing non-zero entries. The relationship between scan pixel size and the convergence angle changes the spatial frequency limit above which no double overlaps occur. The less often spatial frequencies create double overlaps, the smaller the matrix $\mathbf{B}$ becomes. In this study the scan area for live processing was limited to a size of $128\times 128$ and microscope parameters were chosen to keep the matrix size low enough to fit into the GPU RAM.

We concentrated up to this point, solely on the live evaluation in order to facilitate optimisation of experimental parameters during the session. This avoided the need for saving vast amounts of 4D-STEM data that would have required significant disc space due to the continuous nature of the experiments. During the experiments only a few reliable data sets were recorded. To verify the reliability of our approach we compared the post-processing of the recorded data against the live experiment to determine if the inherent parameters used in post-processing such as the Ronchigram position and radius, are in sufficient agreement with those of the live results.

In Fig. 5, the post processing of In₂Se₃ data is shown. Here, the two different atom columns highlighted in Fig. 2 a can be seen in the first moment vector field (Fig. 5 a), in its divergence (Fig. 5 b), and in the HAADF image (Fig. 5 c). The phase of the SSB reconstruction (Fig. 5 d) also yields site-specific contrast, which is more pronounced than in the live imaging result in Fig. 3 d. Recalling that SSB ptychography relies on the weak phase object approximation in eq. (1) that breaks down already at the thinnest of specimen as to a quantitative interpretability, conclusions from relative phases among different atomic sites need to be drawn with great care.

In Figure 6, a comparison between different signals as well as between live imaging and post processing of SrTiO₃ is shown. Strontium titanate enables evaluation of the different imaging modes regarding their capability for simultaneous imaging of light oxygen columns and comparably heavy Sr and Ti oxide columns, adding to the results obtained for In₂Se₃ in Fig. 5. The first moment vector field (Fig. 6 a) predominantly shows the heavy atom columns as sinks. The fact this vector field also contains sinks at the oxygen sites becomes visible in the divergence map in Fig. 6 b, whereas the HAADF signal recorded separately with the conventional annular detector in Fig. 6 c visualises only the heavy-atom sites of Sr and Ti oxide. In the post-processed SSB (Fig. 6d) the oxygen columns can be easily determined simultaneously with the heavy sites. The live SSB (Fig. 6g) has some noise, but the oxygen columns are still visible. Moreover, a comparison of the conventional HAADF in Fig. 6 c with the annular dark field signal generated by a virtual annular detector applied to the 4D-STEM data in Fig. 6 f demonstrates that practically all main contrast mechanisms exploiting low- as well as high-angle scattering can be captured by the 4D-STEM imaging mode.

Earlier we stated that the live ptychographic reconstruction exploits the SSB algorithm, because it is a non-iterative, linear and direct scheme that allows for in-situ processing. However, the weak phase object interaction model given by eq. (1) is a seemingly drastic limitation of this approach. In the analysis of post processing data, it is beneficial to explore whether a ptychographic scheme based more on an advanced interaction model, neglecting computational hardware constraints for the moment, could have been the better choice for live ptychography. To this end, we used the ePIE algorithm to reconstruct the SrTiO₃ data as shown in Fig. 6 e. The standard ePIE implementation clearly does not give a reasonable result, at least for the usual reconstruction settings reported in literature that we used here. To explore this in more detail, the sample thickness was determined to approximately 25 nm by comparing the experimental PACBED with a thickness dependent simulation as in Fig. 7. Consequently, the specimen thickness was far beyond the validity of both the weak phase object approximation and the single-slice model used in ePIE. At first sight it is nevertheless surprising that the latter approach performs worse, since one must consider it more advanced than the weak phase model from the viewpoint of scattering theory. In fact, ePIE tries to iteratively find both the probe and the object transmission function in such a way that the modulus of the Fourier transform of their product agrees best with all details of the experimental diffraction data. When dynamical scattering sets in, this multiplicative interaction scheme is incapable of delivering the details of the diffraction pattern, so that the algorithm does not converge to a reliable solution. In this particular case ePIE has put some lattice information in the probe (Fig. 6 h). This will be studied in the next subsection by means of simulations.

To summarise the experimental results, performing live ptychography using the SSB method had originally been motivated by computational aspects, but contrary to expectations it also turns out to be more robust against the violation of the weak phase object approximation and dynamical scattering. Of course, this only holds for qualitative imaging, but it is a significant advantage in practice where suitable structural contrast is obtained also at elevated specimen thickness for both light and heavy atomic columns.

Partial reconstruction.  

As the implementation of live ptychography based on eq. (3) maps the result of single scan points successively to the final reconstruction, a simulation study has been conducted in which we investigated the accuracy of the reconstructed phase in already scanned regions in dependence of the scan progress. A synthetic dataset has been simulated, based on an SrTiO₃ unit cell as a starting point. Then, a five by five super cell was created by repetition and the phase grating (Fig. 8) has been calculated. Two artificial spatial frequencies were added to the phase grating, one with a wavelength of a single unit cell and one with a wavelength of the super cell. To eliminate dynamical scattering in this conceptual study, a 4D-STEM simulation with $20\times 20$ scan points per unit cell was performed using only one slice with a thickness of one unit cell along electron beam direction $[001]$. Finally, partial SSB reconstructions have been performed using the full range of scan pixels in vertical directions, but only portions of 20 scan pixels horizontally, mimicking a reconstruction during progressive scanning as depicted in Fig. 9. An animation calculated from 100 single pixel columns is available in the supplementary material.

Figure 9 contains the individual blocks of the 20 scan pixel wide reconstructions in the left hand column, and the accumulated result on the right hand side with the scan progressing from top to bottom. Consequently, the phase bottom right is the final reconstruction for the full scan, obtained by our cumulative approach which we found to be identical to a reconstruction using the conventional treatment employing the whole 4D scan. Only here, the low, medium and atomic-scale spatial frequencies are reconstructed correctly without artefacts. In that respect, it is instructive to explore the partial reconstructions in Fig. 9. Since we used subsets that equal the size of a single unit cell, spatial frequencies down to the synthetic one with a period of one unit cell appear at least qualitatively in all partial reconstructions. The sharp edges between the available data and the yet-missing region have resulted in a ringing effect near the edges known as the Gibbs-phenomenon. A closer look at the left column of Fig. 9 exhibits that these artefacts largely interfere destructively during accumulation as seen, e.g. by a maximum at horizontal scan pixel 20 in the top row and a minimum at this position in the row below. Consequently, ringing artefacts become less obvious in the full reconstruction on the right within the region that has already been scanned. Therefore, it is already possible to visualise the atomic structure and partly meso-scale phase variations for partial scans, making it possible to navigate on the sample and visually interpret results in a real experiment.

Real specimens and scan regions do not usually fulfill periodic boundary conditions which still apply to the full scan of Fig. 9. Therefore, Fig. 10 shows the impact of selecting different reconstruction areas by simulating the reconstruction of a smaller scan area that is not aligned with the underlying lattice. SSB reconstructs the specimen with an assumption of a periodic boundary condition and cannot reconstruct spatial frequencies above a certain threshold, as previously discussed. Trying to reconstruct a field of view where wrapping around the edges creates a discontinuity, i.e. frequencies higher than SSB can reconstruct, leads to reconstruction artefacts as seen in Fig. 9. Quantitatively, the difference between the ground truth taken from the marked rectangle of the full reconstruction in Fig. 10 a and a reconstruction that solely employs the scan pixels therein, as seen in Fig. 10 b is mapped in Fig. 10 c and can take significant values of 5-10 % of the phase itself in the present example.

As a solution, the field of view where an accurate reconstruction is required can be surrounded by a smooth transition to a zero-valued buffer area so that the presence of spatial frequencies above the resolution limit is minimized when the reconstruction area is wrapped around at the edges. In future studies a Lanczos filtering (Duchon, 1979) scheme could be added for our live ptychography approach.

Thickness effects.   A comprehensive algorithmic review is not our focus but touching briefly on the findings in conjunction with Fig. 6 we elucidate the impact of dynamical scattering, or, equivalently, specimen thickness, on different signals. A multislice simulation (Rosenauer & Schowalter, 2007) has been performed for SrTiO₃ in $[001]$ projection employing the experimental parameters and using $20\times 20$ scan pixels. The data was evaluated for thicknesses of 1 nm and 30 nm, addressing both the kinematic case and the situation of elevated thickness in our experiment. The results have been compiled in Fig. 11.

Figure 11 a shows the phase grating used in the multislice simulation for structural reference. In figure part (b) we added the theoretical result that would be obtained for SSB ptychography in case all methodological premises were fulfilled in practice. That is, we generated a 4D-STEM data set by means of the weak phase approximation in eq. (1) for a single slice with the thickness of one SrTiO₃ unit cell and performed the SSB reconstruction. Note that this is identical to the phase $\Phi$ in eq. (1), low-pass filtered with a circular aperture that has twice the radius of the probe-forming aperture.

Figures 11 c-j show the results of evaluating the 1 nm (left column) and the 30 nm data (right column) using different methods aligned row-wise. As can be expected from former studies employing first moment based imaging (Müller-Caspary et al., 2017; Müller et al., 2014) of centrosymmetric structures, Figs. 11 c-f resemble the atomic structure in terms of momentum transfer vector maps and their divergences with the atoms being sites of central fields. This is preserved in qualitative manner only for more elevated thicknesses. Note that Figs. 11 c and e are proportional to the probe-convoluted distributions of the projected electric field and charge density, respectively.

Similarly, the SSB reconstructions in Figs. 11 g and h yield reliable structural contrast at both low and elevated thickness despite the violation of the weak phase approximation in eq. (1), which confirms our interpretation in the post processing section. However, already the result in Fig. 11 g should be considered as qualitative except for the oxygen sites. Please note that the probe had been focused on the specimen surface in the simulation. Because the SSB reconstruction considers the 30 nm thick specimen as a single slice here, the optimum focus would have been at some depth inside the specimen which is one reason why atomic sites appear slightly broader (Fig. 11 h).

The situation is different for the ePIE results in Figs. 11 i,j. Whereas the reconstruction for the thin specimen in Fig. 11 i represents the phase excellently and can be considered quantitative within the general framework of validity of ePIE, the algorithm has severe difficulties in reconstructing the object transmission function at 30 nm thickness in Fig. 11 j. In Fig. 11 k the reconstructed probe looks like an airy disc. This is the result of an aberration free probe, limited only by an aperture in diffraction space, and that was used in the simulation. In Fig. 11 l sample information is transferred to the probe. This further confirms our observations concerning ePIE in the post processing section by simulation. However, Fig. 6 e,h gives even less information from the specimen than Fig. 11 j,l, which can be attributed to Poisson noise neglected in the simulation, residual aberrations, and importantly, different manners of separating probe and object which starts to fail when dynamical scattering sets in. To conclude, selecting the SSB scheme for live imaging of structural contrast can also be supported from the simulation point of view.

Low dose.   The performance of the SSB reconstruction and the divergence of the first moment was checked in a low dose simulation (Fig. 12). At 1000 electrons per Ų (Fig. 12a,b) the heavy atom columns are visible. At this dose, the SSB reconstruction gives stronger contrast than the divergence of the first moment. At 10000 electrons per Ų (Fig. 12c,d) also the oxygen atom columns are visible. The SSB reconstruction and the divergence of the first moment shows similar performance at this dose.