A Systematic Methodology to Compute the
Quantum Vulnerability Factors
for Quantum Circuits

Even though the physical implementation of quantum computing technology covers more than 20 different approaches from Quantum dot computers to Nuclear magnetic resonance quantum computers, the most popular and successful attempts in this domain are Trapped Ion and Superconducting quantum computers, the latter being capable of running the majority of quantum algorithms.

As the biggest quantum computing service provider, the IBM quantum system includes 21 quantum computers available, with qubit capacity ranging from 5 to 65, with public access through API services provided by IBM. Among the 21 available quantum computers, three types of quantum processors are implemented: Canary, Falcon, and Hummingbird. Each quantum processor model uses different qubit topologies, which may greatly impact the machine quantum noise and quantum errors on computation results [22].

Quantum programs are expressed as quantum circuits, in which a set of quantum gates are sequentially applied to the initial qubits, and produce a probabilistic output across all combinations of the classical bits. These quantum circuits will be mapped to qubits from Noisy Intermediate-Scale Quantum (NISQ) machines based on the topological connectivity of the qubits and the availability of the supported gate types. Programming on different quantum platforms and quantum simulators has been constrained by the quantum compilers supported by different systems. Qiskit [7] and QASM [23] are generally supported by IBM quantum machines. Both Qiskit and QASM are imperative programming languages and designed machine-independent. In our experiments, we mainly focus on studying the IBM quantum machines and running benchmark quantum programs in Qiskit.

In spite of the fact that the qubit capacity of quantum computers is ever-increasing, the performance of IBM quantum computers is bounded by the quantum noise, which is greatly reducing the accuracy and performance of the IBM quantum computers [24]. Quantum noise can be categorized into operation errors and retention (coherence) errors [25]. A qubit can retain data (position) only for a limited time (coherence time). Retention errors are categorized into two types, T1 and T2 errors [26]. A qubit in a high energy state $\ket{1}$ naturally decays to lower energy state $\ket{0}$, the time associated with this decay is called spin-lattice coherence time (T1). The spin-spin relaxation process (T2) indicates the time for a qubit to get affected by the external environment or by the interaction with other qubits. Depending on the material used in NISQ machines, individual qubits have a time range for T1 and T2 that has improved in the last decade from 1 nano-second to 100 micro-seconds [27].

Hardware-based noise reduction is still under investigation by IBM-Q, which is working to improve the environment and qubit design by updating the quantum processor and even using new isolation techniques. In the IBM-Q system One, IBM tried to isolate the interaction with noise and qubits by separating the quantum computer from the surrounding environment. However, the hardware approach is only one of the dimensions to consider, depending on the circuit depth and noise on gates and measurements, the level of quantum noise affecting the results can change in a certain range.

Quantum Error Correction (QEC) has been fundamental to reach Fault Tolerant Quantum Computers (FTQC). QEC is designed to protect a qubit from noise, at the extremely high cost of requiring from 5x to 9x larger circuits. As particles impacts, unlike noise, are stochastic and unpredictable, unfortunately, QEC is inefficient in handling radiation-induced and correlated faults [14, 15, 13]. To reach FTQC we need to design better QEC, and the first step is the understanding of faults impact and propagation, which is the main scope of this paper.

Preliminary and inspiring works show that ionizing radiation induces faults in superconducting qubits [13, 14, 15, 17] and, once employed in large scale, radiation fault tolerance is expected to be the next big challenge for quantum (super-) computers [9, 10, 11, 12].

As shown in Figure 1, adapted from [14] and [15], the impact of ionizing radiation on the qubit superconducting materials increases the amount of hole-electron pairs in the Aluminum thin-film and Silicon substrate. Heavy particles are more likely to interact with Silicon ($Al$ is transparent to neutrons), other sources of radiation ($\beta$, X-rays) with Aluminum. While there is still no quantitative measurement of the radiation-induced fault rate in quantum circuits, it is known that the additional charge deposited by the impinging particle induces a non-equilibrium that leads to Cooper pairs break and, thus, quasiparticles generation [14, 15]. The resulting excitement modifies the state of the qubit, possibly changing its state (i.e., it induces $\phi$ and/or $\theta$ phase shift) as shown with simulations [15] and experimentally validated [10]. While for CMOS a fault is generated only if the charge is higher than the critical charge [20], on a qubit any excitement modifies the state, inducing a phase shift with a magnitude that depends on the deposited charge [28]. It has also been shown that if, and only if, the deposited charge is sufficiently high the qubit collapses [14].

Terrestrial neutrons and heavy ions, which are today the most critical source of faults for Silicon-based classical computing devices [20], generate a large amount of electrons-holes pairs in the Silicon substrate. The energy spectrum of neutrons ranges from meV to GeV, the lower energy ones being exponentially more common. Unfortunately, the frailty of qubits makes it not unlikely for low energy neutrons and lighter particles, such as muons (almost harmless for CMOS technology [29]), to induce a sufficient perturbation in the qubit to generate a fault [15].

An interesting difference between particles is that $\gamma$-rays, $\beta$, X-rays have a constant and accumulative effect while the neutrons, heavy ions, and muons impact is stochastic and transient. In other words, $\gamma$-rays, $\beta$, and X-rays exposure constantly deposits a little amount of charge, until inducing a fault or qubit collapsing [14] (or a permanent CMOS transistor malfunction [30]). Neutrons and heavy ions strikes are random (the neutrons flux at sea level is $\sim 13n/(cm^{2}\times{h})$ [31]) and the impact with the device material produces a transient charge that can lead to a fault.

Unfortunately, while a thin shielding could be sufficient to drastically reduce the number of X-rays reaching the qubit, as mentioned in [14], the shielding for neutrons and heavy ions is impractical (meters of concrete or lead) and for muons is basically impossible as the qubit should be placed in deep underground caves [15]. As a result, as known for traditional computing devices, it is impossible to shield qubits from transient faults. We must find effective and efficient solutions to deal with the unavoidable transient faults in qubits.

Unlike previous work on quantum fault tolerance, we focus on the effects of faults in quantum computation rather than investigating the fault physical mechanisms. As in the AVF or PVF measurement, we assume that the fault occurred independently of the cause, and understand its effect on the quantum circuit output. While recently a preliminary fault injector to track noise propagation was presented [32], there is still no fault injector to track transient fault propagation in quantum circuits. In this paper, we introduce a novel fault injector integrated with Qiskit and that can be used also in real quantum machines. Additionally, we formalize a new metric, the Quantum Vulnerability Factor, based on Michelson Contrast, to ease the analysis of quantum circuit reliability.

We aim at practically estimating the reliability of a circuit and at identifying the qubits in a circuit that, once affected, are more likely to induce a negative impact on the circuit correctness. Such information is highly valuable as it allows to map the circuit qubits to physical qubits in the most reliable way and to predict the effects of faults in the quantum computation.

The output of a quantum circuit is a set of states, each with a different probability. The state(s) with the higher probability is(are) considered the output. Most works use the Probability of Successful Trial (PST) metric when evaluating the reliability and correctness of a quantum circuit output [33, 34, 35, 36]. PST considers only the probability of the correct state and is defined by Equation 1.

The PST metric on its own cannot quantify nor qualify if a circuit is reliable and cannot give overall information about the state probability distribution we are investigating. To use PST one needs to specify a threshold to define when a circuit can be considered reliable. For instance, one could argue that circuits with $PST>0.68$ are sufficiently good, and a circuit with a $PST=0.5$ fails the reliability assessment. Unfortunately, defining a threshold, such as 1-sigma, masks the details of the fault effects on the output state distribution and may deem reliable circuits as unreliable. As we show later in this Section, using the example in Figure 2, values close to the threshold are identified as correct but can be highly susceptible to external perturbations, such as radiation. While increasing the PST may indeed improve the reliability, in fact, this increase can also affect the overall probability distribution and thus increase the probability of a single incorrect state.

To understand the reliability of a circuit, and consequently the impact of a fault on the output correctness (i.e., on the output probability distribution), we extend the PST metric as follows. First, we compute the Michelson Contrast [37]. We use the Michelson Contrast since it naturally measures how distinguishable one object is from others using color, luminance, or, as in our specific case, the probability of each output state. In other words, the Michelson Contrast defines how confidently (i.e., how distinguished) one can select the correct state among all states in the output.

Equation 2 shows the contrast computation, where $P(A)$ is the probability of the correct state (i.e., the expected state in a fault-free execution), and $P(B)$ is the highest probability among any of the incorrect states (i.e., the most probable incorrect state). Please note that PST is precisely $P(A)$ in Equation 2 of our contrast metric [34], but we include $P(B)$ to consider also the overall probability distribution.

Since it is possible for a circuit to produce $P(A)<P(B)$ (e.g., due to technology, noise, or external factors such as radiation-induced faults), the contrast range is $[-1,1]$. It is worth noting that the contrast metric is not limited to circuits with a single correct state, the extension for multiple-state circuits can be easily performed by aggregating the probabilities of all correct states into $P(A)$.

To shift the range to $[0,1]$ and to have lower values indicating a more reliable configuration (as for AVF and PVF), the QVF is calculated as shown in Equation 3:

By this definition, QVF values close to zero indicate a clear contrast between the correct state and the incorrect ones, with the correct state presenting the highest probability. In other words, the probability to have the expected output state is very high compared to the other states. QVF values around $0.5$ present the correct state and at least one incorrect state with similar probabilities, which makes the identification of correct states dubious. Finally, values close to one represent the worst case in a probability distribution where the correct states are not even as high as the incorrect ones.

To better illustrate the QVF calculation and meaning, Figure 2 shows the probability distribution for a 2-qubits Grover circuit in which we perform 3 fault injections. The correct state is $11$ and the most probable incorrect state is $01$ for all three faults. Then, $P(A)=P(11)$ and $P(B)=P(01)$ for all three fault injection. The $QVF$ results for each circuit are as follows:

• •

  Fault 1: $P(A)=0.949$, $P(B)=0.024$, $QVF=0.03$

• •

  Fault 2: $P(A)=0.484$, $P(B)=0.486$, $QVF=0.50$

• •

  Fault 3: $P(A)=0.361$, $P(B)=0.604$, $QVF=0.63$

The circuit has a very low QVF for the first fault ($0.03$), as evident from Figure 2, indicating that the correct state $11$, with a probability of about 95%, can be reliably selected. The second fault has a QVF of about $0.50$, which indicates that it is not possible to reliably select the correct state ($11$ and $01$ states are almost equally probable). Finally, the third fault has a high QVF of $0.63$, indicating that the incorrect state $01$ is more likely to be selected as the output, which can lead to errors. As a result, we can state that fault 1 (low QVF) is not very critical, while fault 2 and mostly fault 3 should be strictly avoided as they drastically change the output state distribution. It is worth noting that PST would have identified the outcome of fault 1 as correct and both faults 2 and 3 as wrong. However, no information about the impact on the output state distribution would have been given.

To further illustrate the weakness of PST with an arbitrary threshold in the evaluation we are proposing, let us consider a fault in a 5-qubit circuit with the probability distribution shown in Figure 3. The correct state of this circuit is $00100$ and the probability for this state is about 50%, while the probability for any incorrect state is at most 1.76% (i.e., all of the 31 remaining states have a similar probability). Thus, the output of this faulty circuit results in a PST of $0.5$, rejecting that circuit as unreliable (depending on the threshold defined). However, the QVF is $0.03$ indicating that the correct state can be confidently identified, even if one defines a strict threshold. This is evident in Figure 3, since the state $00100$ is the only state with non-negligible probability, and the remaining states can be easily identified as noise.

To measure the QVF as well as the impact of faults, we designed a fault injector for the quantum circuit described in Section 4. We inject faults in each qubit and track the effect of each injection in the output. When comparing the QVF of a fault-free circuit to a faulty one, we can observe how much a circuit or its individual qubits are sensitive to faults.

We model the faults to inject based on the latest discoveries on qubits radiation vulnerability, that have shown that even a small deposition of charge from an impinging particle reduces the coherence time and can lead to unexpected modifications of the qubit state [15, 28, 10, 14].

In a classical computer, a fault is generated when the impinging particle deposits enough charge to change the binary state of a transistor (i.e., the deposited charge is higher than the critical charge [20]). The qubit state is associated with the $\phi$ and $\theta$ coordinates that represent its amplitude and phase. In a qubit, then, the excitement that derives from the impact of the impinging particle is logically reflected as a state change and can be represented with $\phi$ and/or $\theta$ phase(s) shift(s), independently of the source (see Figure 1).

A $\theta$ phase shift changes the 0-1 probability in the qubit, while a $\phi$ phase shift changes its orientation. Both shifts can impact the correctness of the execution, and it is part of our contribution to identify which phase shift in which qubit is more critical for a quantum circuit. Albeit not yet experimentally quantified, it has been mathematically shown that a higher charge deposition induces a bigger phase shift [28]. As the charge deposition of the radiation-induced impact depends on the energy of the impinging particle (which goes from meV to GeV [20]) and on the distance between the impact location and the qubit, the resulting phase shift magnitude can be largely variable. Thus, we inject shifts of different magnitudes as well as combinations of shifts ($\theta+\phi$) to correlate the circuit behavior to a wide spectrum of effects. As discussed in Section 4, the amplitude and direction of the injected phase shift is a parameter in our fault injection framework.

A major event or a cumulative charge deposition, as shown in [14] for X-rays, can also lead the qubit to collapse. In such an event, the qubit ceases to operate, there would be no reason to measure its QVF and there would be no other practical solution than re-executing the whole circuit. As the vast majority of neutrons have low energy and the cumulative effect of X-rays can be easily shielded [14], radiation-induced qubit collapses are expected to be less likely than phase shifts.

The quantum circuit execution is known to be noisy, even in the absence of faults. Faults will happen on top of the already noisy execution. Based on the information provided e.g., by IBM, it is possible to have a clear and detailed characterization of the noise in each physical qubit [38].

To have a full understanding of the effects of faults in the quantum circuit execution, our fault injector can be executed with three different scenarios. (1) Simulation without external noise, ideal but unrealistic environment. (2) Simulation of a physical machine, using the IBM-Q noise model to create a realistic environment based on actual quantum computers. (3) Injection over the circuit execution on a physical IBM-Q machine. In this paper, we only present data obtained with scenario (2) and (3) since scenario (1) cannot be achieved in the real world. Considering the noise, executed or modeled after a physical machine, helps us to understand how each qubit and fault severity can change the state probability distribution. It is worth noting that it is possible for a fault to attenuate the noise effect by changing $\phi$ or $\theta$ in opposite directions, improving the circuit reliability.

Our fault injector is built on the open-source and well-documented Qiskit framework [7]. The fault injector operates over a Qiskit’s QuantumCircuit object to generate new circuits with fault(s) injected. The new faulty circuits can be transpiled and executed just as one would execute a regular QuantumCircuit. Thus, we can execute the faulty circuits in physical IBM-Q machines as well as simulators, or even export them as QASM files to load and execute the circuits on different systems.

To model the injected fault, we used Qiskit’s most generic gate, the U gate, to effectively simulate every possible phase shift. The U gate can be described as:

which receives three parameters:

• •

  $\phi$ is the angle defined in the $XY$ plane of the Bloch sphere (a rotation angle on the $Z$ axis);

• •

  $\theta$ is the angle defined in the plane that includes the $Z$ axis and the vector representing the generic quantum state $\ket{\psi}$;

• •

  $\lambda$ is also a rotation on the $Z$ axis.

The parameters for the $U$ gate have been thus selected:

• •

  $\phi$ = $[0,2\pi[$ each $15{{}^{\circ}}$;

• •

  $\theta$ = $[0,\pi]$ each $15^{\circ}$;

• •

  $\lambda$ = 0

This angles combination results in $312$ possible configurations of the $U$ gate (injections) for each position in the quantum circuit.

For the purpose of this study, we inject faults after each gate in the original circuit, simulating faults in each one of the circuit operations (see Figure 4). We also inject only single qubit faults, which means that for each multiple qubit gate we generate multiple circuits with only one qubit at a time affected by the fault. For instance, a two-qubit CNOT gate will generate two distinct circuits with a fault on only one of the qubits each. It is worth noting that our fault injector is also able to inject (correlated) multiple faults, as these events have already been observed in multiqubits circuits [15]. However, we leave the multiple faults study for future work and in this paper, we limit the analysis to single faults since the scope of this work is to introduce the QVF metric and the fault injection framework, which are independent of specific fault models.

As the radiation-induced phase shifts can be of different magnitudes, we test several phase shifts. For each fault position we inject $312$ possible phase shifts (i.e., transient faults), and, for each fault, we execute the circuit $1,024$ times to obtain the probability distribution. Bernstein-Vazirani has 13 possible fault positions, thus requiring $4,056$ distinct faults and a total of $4,153,344$ executions. Grover and Deutsch-Jozsa have 18 fault positions each, which leads to $5,616$ distinct faults and $5,750,784$ executions. Thus, for the three circuits, we perform a fault injection campaign of $15,654,912$ total executions. While this seems a large number of configurations, the time to generate the faulty circuits and the overhead of executing (or simulating) the faulty circuits are negligible. Once the phase shift is correlated with the particle impact, we can limit the injections to the relevant phase shifts. The original codes of circuits, data of faults injected, and output results are publicly available [21].

Figure 4 illustrates the fault injection in a 2-qubit Grover circuit. The fault is injected in qubit $q_{0}$ after the first Hadamard gate with a $\theta$ phase shit of $\frac{\pi}{4}$. The probability distributions of the original and faulty circuit are plotted on the right. The correct state is $11$ and the $P(A)$ (i.e., the PST) is $0.937$ for the original circuit and $0.814$ for the faulty one. To compute the contrast in Equation 2 and the QVF in Equation 3, we also need $P(B)$. For the original circuit $P(B)=P(10)=0.040$, while for the faulty one $P(B)=P(01)=0.150$. Then, the QVF is $0.04$ for the original and $0.16$ for the faulty circuit.

It is worth noting that the fault positions, as well as the fault model, can be easily modified to better correspond to reality as the understanding of quantum computer’s reliability progresses.

The quantum threshold theorem, as an analogue to von Neumann’s threshold theorem for classical computers [41], states that it is possible to perform arbitrarily long quantum computations on a faulty quantum computer if the error probability per gate is below a certain threshold [42]. Recently, it has been also shown that the poly-logarithmic factor present in the standard threshold theorem is actually not needed and the factor can be reduced to a constant [43]. It appears that there are no physical implementation limitations for quantum computer realization, and this is an exciting result.

However, the threshold theorem needs some physically reasonable assumptions about the type of noise. Therefore, although the theorem demonstrates that a fault tolerance solution exists, such a solution needs to be implemented and is obviously engineered based on the expected error rate to avoid unnecessary overheads. Not considering radiation-induced faults risks to underestimate the error rate and to guarantee sufficient reliability it will be necessary to implement additional qubit fault tolerance [15]. Identifying the qubits that are more likely to require additional fault tolerance becomes then fundamental not to overestimate the redundancy or modification to apply to the circuit to make it sufficiently reliable. QVF can therefore be very useful in order to understand the impact of asynchronous faults on a circuit. This problem will become more and more important with the future increment of available qubit numbers and Michelson contrast’s based QVF definition works well even when, due to a very large number of qubits, it is no more possible to fully simulate the circuit or to run it for a number of times large enough to fully understand its probability distribution function; knowing the correct state probability and the most probable among the wrong states will suffice.

Multi-programming, much like in classical computation, is required to improve throughput and better utilize quantum hardware, especially in NISQ-era quantum chips [44]. However, mapping qubits into physical ones for single or multiple quantum circuits has also an impact on the overall circuit reliability [36]. This impact is caused by the limited high fidelity quantum resources, cross-talk noise, and SWAP operations inserted due to the machine topology [35, 45]. The qubit QVF information we provide can be used also to improve such mappings, even in the absence of faults. To improve the quality of the circuit output, for instance, the qubits that are found more resilient to injections should be allocated to resources with lower fidelity. Recently proposed qubit mapping [35, 45] could further improve the circuit reliability by considering QVF as a mean to identify the qubits that are more likely, in the presence of noise, to affect the output correctness.