Realistic quantum photonic neural networks

Quantum neural networks, brain-inspired quantum circuits, harness artificial intelligence to enhance quantum information processing. When driven with light, quantum photonic neural networks (QPNNs) leverage the strengths of mature photonic platforms [1], including multiplexing, low latency, and ultra-low operational powers already being exploited by conventional neural networks [2] and linear-optical quantum processors [3]. This allows QPNNs to perform quantum state tomography [4], act as quantum simulators [5, 6], process [7] or reduce the noise [8] of quantum states, or speed up tasks normally carried out by classical neural networks, such as image recognition [9] and natural language processing [10].

An example of a QPNN circuit, a two-layer network trained to act as a Bell-state analyzer (BSA), is shown in Fig. 1a.

Here, the connectivity and activation function for the network are provided by linear, rectangular interferometer meshes $\left(\mathbf{U}\right)$ [12] and single-site optical nonlinearities $\left(\boldsymbol{\Sigma}\right)$, respectively. A BSA can distinguish between, or create, all four highly entangled Bell-states, and the addition of this nonlinearity ideally increases the success probability of the circuit to unity [7] from 0.5 as possible solely with linear optics in the absence of ancillary photons [11]. The operation performed by the QPNN in this example is therefore crucial to entanglement swapping [13] and hence provides a route toward a deterministic quantum repeater node [14], a vital component of a future quantum internet [15].

Quantum photonic circuits are not ideal, and here we report on the performance of realistic, imperfect QPNNs. Specifically, we consider how propagation losses and imperfect optical nonlinearities affect the fidelity of the QPNN, using the example of a BSA to benchmark our results. As shown in Figs. 1b and c, we find that even realistic networks with weak nonlinearities can vastly outperform those based on (ideal) linear optics. In Fig. 1c, we observe that using state-of-the-art waveguide fabrication (as described in Sec. II) and a perfect $\left(\pi\right)$ two-photon nonlinearity, 10 BSA nodes made from 2-layer QPNNs can be applied in series with a success rate of $72\%$. Surprisingly, this rate is only decreased to $61\%$, for a much weaker $\left(\pi/4\right)$ two-photon nonlinearity if a third (lossy) layer is added. Moreover, if each operation is conditioned on the measurement of two photons, the conditional success rate of 10 nodes becomes $99.99999\%$ and $99.9\%$ for $\pi$ and $\pi/4$, respectively. In what follows, we unravel the dependence on loss, effective nonlinearity, and network size, providing a methodology for the design of optimal QPNNs with realistic components.

The architecture of a realistic QPNN is the same as that of an ideal network, and is thus designed to operate on dual-rail encoded photonic qubits [7]. Each layer consists of a mesh of tunable Mach-Zehnder interferometers (MZIs) with two controllable phase shifters $\left(\phi,\theta\right)$, as shown in the inset to Fig. 1a. The interferometer mesh can be programmed to perform any arbitrary linear unitary transformation $\mathbf{U}\left(\boldsymbol{\phi},\boldsymbol{\theta}\right)$ on the spatial modes of the photons [12]. Single-site nonlinearities, of strength $\varphi$, are placed between consecutive layers. In the Supplementary Information S1, we provide further details on the construction of the system transfer function.

The components of linear photonic networks are not perfect, and various techniques have been developed to mitigate the effects of these imperfections. Specifically, both imperfect splitting ratios of the directional couplers (DCs) that form the MZIs and imperfectly calibrated phase shifters lead to errors that can be mitigated by optimizing the circuits after fabrication [16, 17, 18]. In contrast, we account for these errors and those due to imbalanced photon loss or imperfect nonlinearities by training the variational parameters $\left\{\boldsymbol{\phi}_{i},\boldsymbol{\theta}_{i}\right\}$ in situ, as would be done on-chip, post-fabrication.

We model a realistic linear mesh by allowing each element to suffer from slightly different imperfections, resulting in unbalanced, photon-path-dependent errors. We define the transmittance of each DC as $t$, randomly selected from a normal distribution with a mean of 50% and standard deviation of 5.08%, matching experimental results of a broadband DC fabricated for silicon-on-insulator (SOI) platforms [19]. Likewise, propagation losses, where photons are scattered out of the circuit due to, for example, surface roughness, or are absorbed by the waveguides, are parameterized by

for an element of length $\ell$ and propagation losses per unit length $\alpha_{\mathrm{WG}}$. These losses depend on the platform upon which the photonic circuit is constructed [20, 21, 22, 23, 24, 25, 26], which additionally determines the size of each photonic element. In our analysis, we select $\alpha_{\mathrm{WG}}$ from a normal distribution with a standard deviation of $6.67\%$ of the mean, as is the case for current state-of-the-art photonic circuits built on SOI, which suffer from $\alpha_{\mathrm{WG}}=0.3\pm 0.02$ dB/cm at 1550 nm [26]. More information on the inclusion of fabrication imperfections can be found in Sec. VI and the Supplementary Information S1.

In our architecture, as in previous realizations [7], a Kerr nonlinearity, resolved in the Fock basis as,

is assumed, wherein the ideal case $\varphi=\pi$ such that a single photon passing through will experience no phase change while two photons will undergo a $\pi$ phase change. To date, a $\pi$ Kerr-nonlinearity has yet to be observed at the single-photon level; however, this efficiency has been reached by other nonlinearities, such as those based on electromagnetically induced transparency [27] or the saturation [28] of atoms. While these enable neural networks capable of quantum state tomography [29] or image recognition [30] respectively, neither are scalable nor compatible with most quantum information processing applications since they lead to photon loss. It is, however, likely that in the near future, efficiencies approaching $\pi$ will be demonstrated, either through the coherent chiral scattering of photons from single quantum emitters [31], or using integrated nanophotonic cavities designed to address this need [32, 33, 34]. Hence, we consider single-site Kerr nonlinearities but examine the performance of the QPNN also in the realistic scenario where $\varphi\lesssim\pi$.

In sum, the total transfer function for an $L$-layer QPNN is given by,

This transfer function will act on any input state to produce an actual output state $\left|\psi_{\mathrm{out,act}}^{(i)}\right\rangle=\mathbf{S}\left|\psi_{\mathrm{in}}^{(i)}\right\rangle$, which is compared to the ideal output state $\left|\psi_{\mathrm{out}}^{(i)}\right\rangle$ to determine the unconditional fidelity for that input-output pair,

The total unconditional fidelity, the chance that the network provides the correct output state for any given input state without preconditions, is then found by averaging over all $K$ input-output pairs according to

Conversely, we can calculate the unconditional infidelity (i.e. network error) according to $\mathcal{C}^{(\mathrm{unc})}=1-\mathcal{F}^{(\mathrm{unc})}$, which we minimize in training the QPNN, using the local gradient-free BOBYQA nonlinear optimization algorithm [35], as available in the NLopt library [36]. Further details on network optimization are provided in Sec. VI.

The success of the network may be conditioned on the detection of photons only in ports which abide by the dual-rail encoding scheme, as in this case we know that a logical output was produced. For the BSA shown in Fig. 1a, this means that a single photon was detected in one of the top two modes and the other in the bottom two modes. We call this the conditional fidelity $\mathcal{F}^{(\mathrm{con})}$, each $i^{\mathrm{th}}$ term of which is related to the unconditional fidelity by the probability that the network operation results in a computational basis state (that is, no photons are lost and each logical photonic qubit contains a single photon at the output) $\mathcal{P}^{(cb)}$ by,

In Sec. VI, we provide the expressions used to calculate these conditional measures. For a network operation that requires detection, such as the BSA and not, for example, the realization of a quantum logic gate for quantum computation, $\mathcal{P}^{(\mathrm{cb})}$ gives the probability that the network successfully performed its task, while $\mathcal{F}^{(\mathrm{con})}$ provides the quality of the result; in practice, one may optimize on either the conditional or unconditional infidelities, depending on the task under consideration. In the following, we train solely on the unconditional, however, we provide results from optimizing the conditional infidelity in the Supplementary Information S2.

We begin to consider the effects of imperfections on QPNNs by holding the nonlinearity at the ideal value ($\varphi=\pi$) but introducing DC splitting ratio variations and photon loss as described above. The resultant unconditional infidelity of a BSA for 2 to 6-layer QPNNs with losses ranging from 0.001 to 3 dB/cm, as a function of training iterations, is shown in Figs. 2a-c.

Each case is repeated 50 times, resulting in plateaus of $\mathcal{C}^{(\mathrm{unc})}$ that increase in value for increasing losses, as expected. Interestingly, for 2-layer BSAs, we observe a large spread in the final $\mathcal{C}^{(\mathrm{unc})}$, particularly for low-loss networks (c.f. blue and purple curves in Fig. 2a), indicating that the final performance of the QPNN is largely dictated by imbalance due to imperfect DCs. Adding more layers to the network, as in Fig. 2b and c, reduces this spread, showing that larger QPNNs may learn to correct for these errors and more often reach optimal performance.

This is reflected in Figs. 2d-f, which show the unconditional fidelity $\mathcal{F}^{(\mathrm{unc})}$ as a function of waveguide loss for different sized networks. Here, we compare the in situ trained QPNNs of Figs. 2a-c, denoted by the symbols, to the case where the network is trained offline. Offline training means that a perfect network was trained, then losses and DC errors were subsequently added to the solution. This was repeated 50 times for each $\alpha_{\mathrm{WG}}$, selecting different random imperfections at each repetition, with the mean given by the solid-black curve and standard deviation by the grey region. When trained in situ, the QPNN learns to overcome these imperfections as is seen by the convergence toward loss-limited performance (see Sec. VI for more information on network training and the loss limit). This is more apparent for larger networks, where the fidelity of those trained offline significantly reduces due to increased losses and DC errors, while those trained in situ maintain and even increase $\mathcal{F}^{(\mathrm{unc})}$.

The balance between fabrication imperfections and network size, as a function of losses, is summarized in Fig. 2g. Here, we observe that for state-of-the-art losses $\left(\alpha_{\mathrm{WG}}=0.3\text{ dB}/\text{cm}\right)$ or worse, the unconditional fidelity decreases as expected when more layers are added to the network. When $\alpha_{\mathrm{WG}}=0.3\text{ dB}/\text{cm}$, $\mathcal{F}^{(\mathrm{unc})}\geq 0.905$ (0.904), where the bracketed result is the lower bound of the 95% confidence interval, even for a 6-layer QPNN, demonstrating that high-efficiency performance is possible on realistic state-of-the-art systems. Conversely, a more complex evolution is seen in Fig. 2g for 0.01 dB/cm losses or less, where there exists an optimal network size other than 2 layers. Losses at 0.01 dB/cm are similar to those of the silicon nitride platform for 1550 nm, reported as low as 0.007 dB/cm [37], but more typically near 0.01 dB/cm [38, 39, 40]. In this low-loss case $\left(\alpha_{\mathrm{WG}}=0.01\text{ dB}/\text{cm}\right)$, $\mathcal{F}^{(\mathrm{unc})}$ first increases from 0.993 (0.949) to 0.998 (0.997) by adding two layers to the base size, as the network is better able to account for imperfections. The unconditional fidelity then only slightly decreases to 0.996 (0.996) as the network grows to 7 layers. That is, near-deterministic QPNN-based quantum elements such as BSAs will be realistic in the near-future as platform losses continue to decrease.

The situation is even more promising if the success of the network is preconditioned on detection in the computational basis, as is shown in Figs. 2h and i. Here we present $\mathcal{F}^{(\mathrm{con})}$ and $\mathcal{P}^{(\mathrm{cb})}$ for different sized QPNNs, and for differing $\alpha_{\mathrm{WG}}$. Even for the extremely lossy networks, where $\alpha_{\mathrm{WG}}=2\text{ dB}/\text{cm}$, $\mathcal{F}^{(\mathrm{con})}$ remains above 0.9999 (0.9888) for all $L\leq 7$, while for state-of-the-art losses this conditional fidelity does not drop below 0.999999 (0.999784) for $3\leq L\leq 7$, as we observe in Fig. 2h. In fact, as shown in Fig. 2i, it is mainly the rate at which the QPNN produces a logical output that is affected by an increase in network size, showing the potential of even lossy networks if fault-tolerant protocols are used.

Having studied the effect of fabrication errors on network performance, we now turn to the consequences of sub-optimal nonlinearities. Assuming state-of-the-art losses $\left(\alpha_{\mathrm{WG}}=0.3\text{ dB}/\text{cm}\right)$, we vary the effective nonlinear phase shift $\varphi$ from the ideal $\pi$ to $\pi/100$ and attempt to train QPNNs of different sizes to act as BSAs (see the Supplementary Information S4 for exemplary training traces, c.f. Figs. 2a-c.) For each network size and effective nonlinearity, we attempt to train 200 QPNNs, showing the results in Fig. 3.

Figs. 3a-c depict the highly non-trivial dependence of $\mathcal{F}^{(\mathrm{unc})}$ on the effective nonlinearity $\varphi$. When the QPNN is trained offline, $\mathcal{F}^{(\mathrm{unc})}$ increases monotonically with $\varphi$, as would be the case for a quantum-optical Fredkin gate-based BSA [41, 42, 43] (see the Supplementary Information S4 for additional information). Conversely, a QPNN can be trained to account for the weak nonlinearity, in which case it can vastly outperform this expectation. Considering a 2-layer network (Fig. 3a), we observe a strikingly different $\varphi$ dependence when comparing the best-case in situ trained networks (triangles; see the Supplementary Information S3 for statistical analysis information) to both the average of all successful in situ training cases (circles) and networks trained offline. We observe that networks trained in situ can reach the loss limit with sub-optimal nonlinearities, in addition to fabrication imperfections. Specifically, we observe optimal performance of 2-layer QPNNs when $\varphi=\pi/2$ in addition to $\pi$. Moreover, as can be seen in Fig. 3a, near-optimal performance is reached for a domain of $\varphi$ centred at $\pi/2$, providing a pathway to robust QPNN-based BSAs without the need for a perfect Kerr nonlinearity. It must be noted, however, that operating with weaker nonlinearities decreases the probability that the QPNN converges at the loss limit during training, as is shown in the Supplementary Information S4.

For all $\varphi$, a QPNN trained in situ learns how to account for weak nonlinearities and thus approach the loss limit. These capabilities improve as redundancies are added via an increase in network size, as visually evident across Figs. 3a-c and summarized in d. By adding a single additional (lossy) layer, QPNNs were trained to within $1.13\%$ of the unconditional fidelity achieved with the ideal nonlinearity, 0.951 (0.950) at $\varphi=\pi$, and within $1.15\%$ of the loss limit, 0.952, for all $\varphi\geq\pi/4$. Even networks with nonlinearities as weak as $\varphi=\pi/10$ approach the loss limit at 6 layers, in contrast to the case of $\pi/100$ where $\mathcal{F}^{(\mathrm{unc})}$ increases only to 0.528 (0.528) at 7 layers from 0.499 (0.498) at 2 layers, essentially acting as a linear-optical BSA [11]. Trainability also improves with increased network size, as it becomes easier for the QPNN to find optimal solutions, such that the average unconditional fidelity achieved during in situ training approaches the maximum plateau.

In Figs. 3e and f, the conditional fidelity and computational basis probability are shown as a function of $L\leq 7$, for differing $\varphi$. In contrast to the case where photon losses were varied (c.f. Fig. 2), we observe that the behavior of $\mathcal{F}^{(\mathrm{con})}$ strongly depends on $\varphi$. While QPNNs with $\varphi=\pi$ and $\pi/2$ operate with $\mathcal{F}^{(\mathrm{con})}\geq 0.9999\left(0.9998\right)$ for all $L\leq 7$, networks with nonlinearities near $\pi/4$ and $3\pi/4$ require at least 3 layers to reach this level, while at $\pi/10$, 6 layers are needed. For all nonlinearities and network sizes, $\mathcal{P}^{(\mathrm{cb})}$ is within 0.009 (0.031) of loss-limited performance, as seen in Fig. 3f, and as expected for a QPNN suffering from state-of-the-art losses (c.f. Fig. 2d-f and i). Altogether, this demonstrates that for each combination of fabrication imperfections and effective nonlinearity, there exists an optimal network size that maximizes $\mathcal{F}^{(\mathrm{unc})}$. While adding layers will always tend to increase $\mathcal{F}^{(\mathrm{con})}$, a balance must be struck with the exponential decrease in $\mathcal{P}^{(\mathrm{cb})}$. In the Supplementary Information S5, we demonstrate a QPNN trained to generate Greenberger-Horne-Zeilinger states, and show that this remains true beyond the BSA application.

We have shown that high-fidelity operation is possible in realistic quantum photonic neural networks based on non-ideal Kerr nonlinearities. Since propagation through these networks leads to inevitable photon loss, their unconditional fidelity ceiling tends to decrease with increasing size. While this loss limit is unavoidable, these networks are able to learn to manage additional errors from non-uniform losses and directional coupler splitting ratio variations, often demonstrating increased fidelity with the addition of imperfect layers. Crucially, we have shown that weak nonlinearities, which are mere fractions of the ideal, are sufficient for near-optimal network performance. Even as these sub-optimal nonlinearities are realized [31, 32], the desired phase change will likely be accompanied by wave-packet distortions [44, 45], and although complex solutions based on dynamically-coupled cavities have been proposed [33, 34], it remains an open question if, instead, a QPNN may learn to overcome them in much the same way it does fabrication imperfections. Already in the work presented here, QPNNs offer a fascinating view of the intricate balance between loss, imperfect photon routing and weak nonlinearity, which we have unravelled to demonstrate how each combination leads to an optimal network geometry. Understanding and respecting this balance will be important, in the near future, as realistic QPNNs are designed and fabricated.

It is now clear why QPNNs far outperform linear-optical networks. Even with a weak $\pi/4$ effective nonlinearity, they can learn to surpass the 0.5 unconditional fidelity possible with perfect linear optics [11], achieving $\mathcal{F}^{(\mathrm{unc})}=0.820$ (0.809) at 2 layers (see Figs. 1b and 3a), which grows to 0.951 (0.949) with an additional layer (see Fig. 3b). At 6 layers, loss-limited operation, $\mathcal{F}^{(\mathrm{unc})}=0.891$ (0.890), can be achieved with nonlinearities as weak as $\pi/10$. Returning to Fig. 1c, which summarizes the success rate of operating $N$ BSAs in series, as would be necessary to connect quantum repeater nodes by entanglement swapping [13, 14], the performance benefits offered by QPNNs become more apparent. While 10 consecutive perfect linear-optical BSAs have a success rate of just 0.1%, 6-layer, $\pi/10$ nonlinearity QPNNs reach 31.5%, and 3-layer, $\pi/4$ networks achieve 60.5%.

Preconditioning the success of each QPNN-based BSA on the detection of 2 photons in the correct ports, as would be the case for generating cluster states from fusion gates [46], allows the much higher conditional fidelities to be leveraged. While $\mathcal{F}^{(\mathrm{con})}$ for a perfect linear-optical ten-BSA sequence remains at 1, realistic QPNNs of $\pi/4$ (3 layers) and $\pi/10$ (6 layers) nonlinearities reach 0.999 (0.997) and 0.99999 (0.99996), respectively. Given that these conditional fidelities are all near-unity, the rather large variations to $\mathcal{F}^{(\mathrm{unc})}$ seen above can be attributed to the operational rate of the circuits (c.f. Eq. 6), which are $315\times$ improved when the perfect linear-optical BSAs are replaced by even 6-layer, $\pi/10$ QPNNs. Hence, imperfect QPNNs are likely to play a key role in emerging large-scale quantum technologies.

This research is supported by the Vector Scholarship in Artificial Intelligence, provided through the Vector Institute. The authors thank N.R.H. Pedersen for his insights into linear meshes, and gratefully acknowledge support by the Natural Sciences and Engineering Research Council of Canada (NSERC), the Canadian Foundation for Innovation (CFI), and Queen’s University.

N.R. and J.C. conceived the project, which they developed along with J.E. J.E. was responsible for designing and performing all simulations and analysis, with supervision from B.S. and N.R. All authors discussed the results and shared in the writing and editing responsibilities for the manuscript.