Perspective: Reproducible Coherence Characterization of Superconducting Quantum Devices

Corey Rae H. McRae coreyrae.mcrae@colorado.edu Department of Physics, University of Colorado, Boulder, Colorado 80309, USA National Institute of Standards and Technology, Boulder, Colorado 80305, USA Boulder Cryogenic Quantum Testbed, University of Colorado, Boulder, Colorado 80309, USA Gregory M. Stiehl Rigetti Computing, Berkeley, California 94710, USA Haozhi Wang Laboratory for Physical Sciences, University of Maryland College Park, College Park, MD 20740, USA Sheng-Xiang Lin Department of Physics, University of Colorado, Boulder, Colorado 80309, USA National Institute of Standards and Technology, Boulder, Colorado 80305, USA Boulder Cryogenic Quantum Testbed, University of Colorado, Boulder, Colorado 80309, USA Shane A. Caldwell Rigetti Computing, Berkeley, California 94710, USA David P. Pappas National Institute of Standards and Technology, Boulder, Colorado 80305, USA Josh Mutus Rigetti Computing, Berkeley, California 94710, USA Joshua Combes Department of Electrical, Computer, and Energy Engineering, University of Colorado, Boulder, Colorado 80309, USA

Abstract

As the field of superconducting quantum computing approaches maturity, optimization of single-device performance is proving to be a promising avenue towards large-scale quantum computers. However, this optimization is possible only if performance metrics can be accurately compared among measurements, devices, and laboratories. Currently such comparisons are inaccurate or impossible due to understudied errors from a plethora of sources. In this Perspective, we outline the current state of error analysis for qubits and resonators in superconducting quantum circuits, and discuss what future investigations are required before superconducting quantum device optimization can be realized.

Superconducting quantum computing is poised to become a successful platform for large-scale quantum computing due to promising qubit performance, ease of multiqubit coupling, and potential for scalability. (Gambetta, Chow, and Steffen, 2017) Currently, superconducting quantum computing is transitioning to a mature field of research with efforts from large industrial and governmental organizations as well as many academic groups. Architectures of more than 50 qubits have already been demonstrated. (Arute et al., 2019) However, cryogenic single-photon microwave losses, which limit coherence of individual qubits and resonators, are a performance bottleneck for these systems, (Müller, Cole, and Lisenfeld, 2019; McRae et al., 2020) and efforts to mitigate these losses have been curtailed by understudied errors and unreported experimental details that restrict the accuracy of interlaboratory comparisons.

Refer to caption — Figure 1: Summary of thin film superconductor loss measurements as a function of publication year. Note: Values are not directly comparable due to differences in resonance frequency, $Q_{i}/Q_{c}$ matching, fabrication, shielding, microwave chain, thermalization, and data fitting. (a) Coplanar waveguide (CPW) resonator loss of the best performing "hero" devices, and (b) the same loss multiplied by CPW gap width for normalization purposes. Solid markers: resonator-induced intrinsic TLS loss $F\delta_{\mathrm{TLS}}^{0}$ and $F\delta_{\mathrm{TLS}}^{0}\times s$ , where $s$ is CPW resonator gap width in $\mathrm{\mu}m$ . Open markers: $\delta_{\mathrm{LP}}$ and $\delta_{\mathrm{LP}}\times s$ values, where $\delta_{\mathrm{LP}}$ is low power loss. Superconducting microwave resonators patterned from Al (circles), TiN (squares), and Nb (diamonds) are shown. Marker color denotes reference in which each measurement is reported, with associated colors and references shown in Ref. 4. Original data can be found in Refs. 5; 6; 7; 8; 9; 10; 11; 7; 12; 13; 14; 15; 16; 17; 18. Grey arrow is a guide to the eye showing a decrease in measured loss over time. Grey line denotes the highest performance seen for on-chip superconducting microwave resonator materials loss.

The optimization of individual device performance requires the ability to accurately and precisely compare performance metrics both within a single lab (i.e., A/B device comparisons) and among labs (i.e., comparisons of published measurement outcomes). However, previous experiments have shown that device performance fluctuates in time and among experimental setups as a function of a variety of parameters, many of which are poorly understood. As well, efforts to increase device performance have historically drawn conclusions from the experimental characterization of a small number of devices, and sometimes a single measurement of a single device, due to the large investment of time and money required for a single device measurement.

At present, it is hard to argue that a particular experiment has truly seen a difference in performance, or whether the results are effectively unchanged within the uncertainty of the experiment, since the magnitude of uncertainty in superconducting quantum device performance measurements is largely unknown.

Consider a simple question: Does a certain surface treatment increase or decrease device performance? In theory, two identically-designed circuits would be fabricated, one of which is exposed to the surface treatment and one which is not. Then, the samples would be cooled down in a dilution refrigerator to $\sim$ 10 mK and performance would be measured. The responses would be analyzed, and the performance metrics would be reported for each device. If the performance of the treated sample is better than the untreated sample, we can conclude that the sample treatment increased the device performance. If the difference in values is within the uncertainty of those values, then we can conclude that the effect of the surface treatment is smaller than the precision of the measurement or is negligible.

In reality, these comparisons are not so simple. While the fitting uncertainty of performance metrics for qubits and resonators is generally knowable and small, the metrics themselves vary significantly within a single cooldown, (Burnett et al., 2019; Schlör et al., 2019; Klimov et al., 2018) from cooldown to cooldown, (Burnett et al., 2019; Schlör et al., 2019; Calusine et al., 2018) among nominally identical devices on a single chip or wafer, (Burnett et al., 2019; McRae et al., 2021; Calusine et al., 2018) and among nominally identical devices from different fabrication runs. (Woods et al., 2019) The energy relaxation time $T_{1}$ of a single qubit in a nominally fixed environment can vary by $\sim$ 50 $\%$ over a period of hours. (Klimov et al., 2018) So before we can address the question of which surface treatment is better, we must first address the question: What is the uncertainty of our reported performance metrics, and how can we minimize this uncertainty to facilitate performance metric comparisons while still allowing the experiment to be practically implementable?

Fig. 1 shows the importance of these types of A/B device comparisons to the future of superconducting quantum computing as a research field. In Fig. 1 (a), resonator loss is plotted as a function of year of publication for the three most common superconductors: Al, TiN, and Nb. A downwards trend in loss is seen for the highest-performing devices, indicating that the field is progressing toward devices with lower loss and thus higher performance. However, when device design is accounted for by multiplying by the coplanar waveguide resonator gap $s$ in order to estimate the filling factor of the TLS material (Fig. 1 (b)), this trend disappears and we see no significant trend in progress in materials loss over the past decade, indicating that the increasing performance trend in (a) could be attributed to design changes. This demonstrates that materials loss mitigation may truly be an area of untapped potential in the effort towards large-scale superconducting quantum computing. On the other hand, the scatter and lack of error bars on these data points indicate a fundamental flaw in experimental design in the field: the accuracy and precision of loss measurements have historically been insufficient to drive improvement.

In this Perspective, we will outline the standard performance metrics and what is currently known about fluctuations in performance for superconducting qubits and resonators. We will also highlight areas of uncertainty that must be filled in before optimization of superconducting quantum device performance can be achieved. This article is not intended as a review or tutorial for superconducting qubits or resonators; readers interested these types of articles are directed towards Refs. 24; 25; 4; 26.

The major sources of error and performance fluctuation that hamper reproducibility studies are schematically depicted in Fig. 2. Conceptually, we can organize these into groups of sources that apply to different experiments. For example, the comparison of performance of a device before and after some materials treatment requires accounting for measurement and analysis error (Sec. III), fluctuations within one cooldown (Sec. IV), and cooldown-to-cooldown variation (Sec. V). An interlaboratory comparison of a single device requires that all above error sources as well as setup-to-setup variation (Sec. VII) be addressed. For a comparison of devices on two separate chips, the above error sources as well as the impact of microwave packaging (Sec. VI) and variation due to fabrication (Sec. VIII) will play a part.

In this work, we assume 2D transmon and 2D resonator implementations, the two primary components of planar superconducting quantum circuits. However, much of the error analysis described here applies to other superconducting quantum devices as well. It is the hope that this Perspective will facilitate measurements of metrics that are both self-replicable and reproducible among laboratories.

I Introduction to Performance Metrics

This section outlines the common performance metrics used to characterize qubits and resonators. Performance metrics of qubits and resonators are affected not only by intrinsic sources such as materials losses but also by aspects of the experimental setup such as microwave components and thermalization, as well as the data analysis process. Here, we assume measurements occur in the quantum computing regime of $\sim$ 10 mK temperatures, single photon powers, and zero applied magnetic field, unless otherwise stated.

I.1 Resonator Performance Metrics

The internal quality factor $Q_{i}$ of a resonator is the ratio of resonator energy stored over energy lost to the environment per cycle and can be determined from a measurement of the scattering parameters. The internal quality factor is inversely related to loss $\delta$ as $Q_{i}^{-1}\simeq\delta$ , and loss from various sources are linearly additive. Due to this additive property as well as the variety of power- and temperature-dependent behaviors exhibited by loss sources, resonator loss is a convenient tool for distinguishing between loss sources. In particular, we can determine the contribution to loss from two-level systems (TLS), the dominant source of loss in state-of-the-art superconducting quantum circuits.

The intrinsic TLS loss $\delta_{TLS}^{0}$ is also used as a performance metric. (McRae et al., 2020) A lower value of $\delta_{TLS}^{0}$ indicates better performance materials and/or interfaces. $\delta_{TLS}^{0}$ can be extracted from power-dependent resonator measurements provided the geometry-dependent filling factor $F$ (De Graaf et al., 2018) of TLS–contributing material is known. In practice, $F$ is difficult to determine due to design complexity, so the product $F\delta_{TLS}^{0}$ is reported instead. This is done by sweeping power, i.e., the average photon number in the resonator, and fitting to the total loss. (Gao, 2008; Richardson et al., 2016) More information on accurate loss measurements of superconducting microwave resonators can be found in Ref. 4.

I.2 Qubit Performance Metrics

The performance of a single superconducting qubit is typically characterized by the characteristic times for energy relaxation $T_{1}$ and transverse relaxation $T_{2}$ or pure dephasing $T_{\phi}$ , where $1/T_{2}=1/2T_{1}+1/T_{\phi}$ . The longer these times, the better the performance of the qubit. These parameters are measured using well-calibrated single qubit pulses. For a full description of $T_{1}$ and $T_{2}$ measurement protocols, see Ref. 24. The ability to distinguish between relaxation and pure dephasing is critical to the understanding and eventual mitigation of qubit noise factors. In addition, the value of $T_{1}$ when the relaxation is dominated by intrinsic materials factors, rather than those induced by the experimental setup, is ideal for reporting in qubit optimization experiments. In this work, we refer to this value as the materials-induced $T_{1}$ .

In order to understand the spectral environment of the qubit in a holistic way, we need to be able to quantify the effects of coherent TLS within some range near the idle qubit frequency. A useful metric would be one which takes into account both the coupling strength $g$ of individual coherent TLS to the qubit as well as the total number of coherent TLS in some probed frequency bandwidth which is ideally the total tunable frequency bandwidth of the qubit. $g$ can be determined for strongly-coupled coherent TLS by measuring the avoided crossing of the transmon with the TLS defect, as shown in Fig. 3. Here, this information is obtained by performing a qubit spectroscopy experiment with the XY drive as a function of applied magnetic flux to tune the transmon $f_{01}$ transition. When this transition interacts with a nearby defect within the tunability band of the qubit, avoided crossings can be resolved. The avoided crossing splitting is related to the coupling by 2 $g$ .

A metric that describes the coupling strength and number of coherent TLS would allow us to place a value on the bandwidth cleanliness of a tunable qubit. By combining time series datasets of $T_{1}$ and $T_{\phi}$ in addition to this metric, we can obtain a more descriptive set of metrics for the general coherence of a qubit.

II Error Analysis of Performance Metrics

When comparing two devices or measurements, many sources of random and systematic error contribute to the reported uncertainty. The magnitudes of all of these errors must be taken into account for fair device and process comparisons. Thus, a full accounting of the total error is required in order to understand the factors contributing to variation in performance metrics in superconducting quantum circuits.

As illustrated in Fig. 2, errors can be organized into seven categories: Measurement and analysis error, fluctuations within one cooldown, cooldown-to-cooldown variation, impact of sample box, setup-to-setup variation, cross-wafer variation, and variation between fabrication runs. Depending on the details of the experiment, some or all of these error types may be relevant. For example, an intralaboratory comparison of two devices may not induce setup-to-setup variational errors. In the following sections, a non-comprehensive overview of error of each type is explored for both qubits and resonators.

III Measurement and Analysis Error

In order to obtain performance metrics for qubits and resonators, information about the underlying distributions is extracted from a dataset. This process can induce errors in a number of ways.

III.1 Resonator Measurement and Analysis Error

Error induced by the vector network analyzer during resonator S-parameter measurements is generally negligible, and fit uncertainty for fitting both $Q_{i}$ and $F\delta_{TLS}^{0}$ is knowable and low (see Table 1 for summary).

Anecdotally, preprocessing and fitting methods have been shown to strongly affect resonator $Q_{i}$ and $F\delta_{TLS}^{0}$ , with an error magnitude dependent on resonator coupling and impedance mismatch (Khalil et al., 2012) among other factors. Future research is required in this area to pinpoint the low-error measurement regime as well as characterize the error in both optimal and non-optimal measurement regimes.

III.2 Qubit Measurement and Analysis Error

Single-qubit pulses are implemented when performing $T_{1}$ and $T_{\phi}$ characterization. Systematic pulse errors can affect measurement outcomes, but the exponential nature of the decays mean these effects are irrelevant. Moreover, sample-efficient and Heisenberg-uncertainty-limited estimation schemes exist for calibrating such pulses. Kimmel, Low, and Yoder (2015)

Measurements of qubit $T_{1}$ and $T_{\phi}$ are largely insensitive to changes in analysis as well as minor changes in experimental data acquisition such as modifying the number of points taken to fit to a $T_{1}$ curve or the number of shots per point, as fitting to an exponential model is more forgiving than the Lorentzian model used to fit $Q_{i}$ . Nevertheless, the estimation of both parameters can be sensitive to drift in the readout quadrature over the time scale of a single experiment.

Errors in potential spectral cleanliness metrics can be induced when identifying TLS and distinguishing them from other signatures in qubit data. The identification of TLS can be performed in multiple ways depending on the data used.

In Ref. 21, TLS are identified manually from data such as in Fig. 6 and their $g$ and $f_{i}$ are determined by fitting to an energy-relaxation model consisting of a sum of Lorentzians, as given in Ref. 31. Alternatively, avoided crossings can be characterized as in Fig. 3. The automation of these TLS detection processes could help avoid user-induced error.

IV Fluctuations Within One Cooldown

Within a single cooldown, the performance metrics introduced in Section I fluctuate in time with varying time scales. Several mechanisms have been linked to or are suspected to cause temporal fluctuations, including ambient electronics temperature fluctuations Staley ; Valery, Hill, and Poletto , TLS loss, (Gao, 2008; Pappas et al., 2011) quasiparticle loss, (Zmuidzinas, 2012; Serniak et al., 2018) and magnetic vortex loss. (Song et al., 2009) Regardless of cause, these fluctuations necessitate the use of time series analysis. In this section, we outline some physical mechanisms, as well as discuss the measurement, analysis and characterization of time series data, with the aim to report typical values and understand temporal and spectral correlations.

IV.1 Time Series Analysis

If some performance metric $P$ changes in time, the value and the uncertainty of $P$ at some particular time is not sufficient to characterize the full behavior of that device. To quantify and analyze fluctuations of a performance metric $P$ within a single cooldown, measurements at many different times $t_{n}$ are needed, which are samples from a stochastic process $P(t_{n})$ (Draščić, 2016). The time between successive samples, known as the sampling period, determines the minimum and maximum frequency component of $P(t)$ that can be estimated from the time series data.

Time series data can then be processed in many ways that illuminate different aspects of the physical sources of fluctuation. Common tools include auto- and cross-correlation functions, power spectral densities, (Kay, 1988) causal models, (Pearl, 2009) process control charts, (Heckert et al., 2002; Box et al., 2005) and Allan variance methods. (Guerrier, Molinari, and Stebler, 2016; Siraya, 2020) These tools can be diagnostic (e.g. detect and classify an instability as drift or diffusion in the experiment via statistical control charts), qualitative (e.g. power spectral densities Klimov et al. (2018); Burnett et al. (2019); Moeed et al. (2019) and Allan variance charts Burnett et al. (2019); Moeed et al. (2019) which can point to certain noise sources). Quantum causal modellingCosta and Shrapnel (2016); Milz and Modi (2021) is being applied to gate level characterization of superconducting devices to identify causal structure. White et al. (2021). Using these tools, time series data can be leveraged to understand dynamics and to assess overall performance.

When taking and analyzing time series data, it is important to keep in mind that performance metrics depend on external parameters other than time. For example, the measured $T_{1}$ of a tunable qubit is both time- and frequency-dependent, so the resulting time series is multidimensional $P(t,f)$ , (Klimov et al., 2018) as in Fig. 6.

IV.2 Two-level Systems

The dominant source of both decoherence and noise in state-of-the-art superconducting quantum circuits is two-level systems (TLS). (Simmonds et al., 2004; Müller, Cole, and Lisenfeld, 2019) TLS populations have a large distribution of energies and tunneling rates, and can cause loss in two main ways: through incoherent, off-resonance populations of TLS that are weakly coupled to the device ("incoherent TLS loss"), and through near- or on-resonance coherent individual TLS ("coherent TLS loss"). Interactions between TLS cause fluctuations in induced relaxation and dispersive shifts, leading to parameter fluctuations in qubits ( $T_{1}$ , $T_{2}^{R}$ , and $f_{01}$ ) and resonators ( $f_{0}$ ). (Müller et al., 2015; Schlör et al., 2019; Béjanin et al., 2021)

Incoherent TLS are fluctuators switching incoherently between two eigenstates, and are the dominant source of low-frequency environmental noise affecting both qubits and resonators. Incoherent TLS loss is effectively background loss that is present in the entire 4-10 GHz frequency bandwidth used in superconducting quantum circuits.

In addition to incoherent TLS, qubits can also couple to coherent TLS located in strong electric field regions, such as Josephson junctions, and resonantly absorb energy from them. This leads to loss at distinct frequencies as well as dispersive shifts which is reflected in $T_{1}$ and $T_{2}$ as well as qubit frequency $f_{01}$ . (Schlör et al., 2019) For strongly-coupled TLS, anticrossings are induced as shown in Fig. 3.

IV.3 Resonator Fluctuations Within One Cooldown

Over a single cooldown, fluctuations in resonator parameters such as resonance frequency $f_{0}$ and time-dependent phase noise are seen as well as in $Q_{i}$ (Fig. 4). It is theorized that TLS loss is the single mechanism that contributes to high-frequency fluctuations in performance of planar resonators within a single cooldown. (Moeed et al., 2019)

In addition to TLS-induced noise, sample thermalization over time strongly affects resonator performance. As the sample thermalizes to base temperature, resonator $Q_{i}$ will decrease, and a shift in resonator frequency $f_{0}$ will be seen as well. This can be attributed to TLS being saturated at high temperatures but becoming available for device interaction at cooler temperatures. (Pappas et al., 2011; Gao et al., 2008) For accurate results, resonator characterization must occur after full thermalization of the sample to base temperature.

Two techniques are commonly used to perform TLS loss measurements. The first technique measures the loss as a function of power. This can be lengthy (on the scale of many hours) when implementing a power sweep to very low, single-photon powers in order to extract the resonator-induced intrinsic TLS loss. Thus, the power sweep method generally does not capture high-frequency fluctuations of TLS loss. However, TLS loss fluctuations over days or weeks have yet to be studied as well.

The second technique is to probe with a fixed high-power tone and vary temperature. This can decrease the measurement time to minutes per point, with additional time needed for sample thermalization at each temperature. (Pappas et al., 2011) One important caveat is that the temperature sweep method has been demonstrated to give resonator-induced intrinsic TLS loss values that can be 20-30 $\%$ higher than the those extracted from the power sweep method. The current understanding of this discrepancy is the temperature sweep method probes a larger frequency range and thus is sensitive to a larger population of TLS. (Pappas et al., 2011) This is an area of study that requires further exploration.

IV.4 Qubit Fluctuations Within One Cooldown

Qubit $T_{1}$ and $T_{\phi}$ , as well as qubit frequency $f_{01}$ , vary significantly over time (Müller et al., 2015; Klimov et al., 2018; Burnett et al., 2019; Schlör et al., 2019; Carroll et al., 2021) (Fig. 5). For $T_{1}$ in particular, extreme fluctuations can occur on the order of tens of seconds and can be up to an order of magnitude in size. (Klimov et al., 2018) Using evidence from qubit time series and spectral data (Fig. 6), the largest of these fluctuations can be attributed to individual coherent TLS that move in and out of resonance with the qubit. (Béjanin et al., 2021; Carroll et al., 2021; Tersoff and Hannon, 2021) A recent study suggests that the few most coherent TLS correspond to more than 10 $\%$ of the total loss in standard qubits. (Tersoff and Hannon, 2021) In fact, temporal fluctuations are significant enough that superconducting quantum computing companies periodically take their remotely-accessible devices offline to re-calibrate due to random and systematic drifts in system parameters. (Staley, ; Valery, Hill, and Poletto, ) Indeed, IBM’s cloud-accessible quantum processors must be regularly benchmarked and recalibrated in order to manage the effects of these TLS on system performance. (IBM, )

Frequency-tunable qubits can be scanned over both frequency and time, giving a descriptive picture of the spectral environment of the qubit (Fig. 6). As well, fixed-frequency qubits can be weakly frequency-tuned by implementing off-resonant microwave tones to drive AC-Stark shifts. (Carroll et al., 2021)

Ref. 51 indicates that qubit $T_{1}$ averaged over a nine-month period can be well-described by $T_{1}$ averaged over a much shorter time period if also implementing tuning over the immediate qubit frequency bandwidth. This suggests that while individual coherent TLS dynamics do strongly affect qubit performance, this effect can be taken into account by probing the near-frequency environment of the qubit. Results in this work indicate that time and frequency bandwidth scan parameters of $\sim$ 1-2 days and $\sim$ 5 MHz can give good correlation to long-term qubit performance.

Frequency scans (Klimov et al., 2018) show sustained lowered $T_{1}$ at qubit line mode and microwave carrier frequencies. They also indicate frequencies of resonant relaxation that can be attributed to coherent TLS. (Béjanin et al., 2021; Carroll et al., 2021; Tersoff and Hannon, 2021) When reporting $T_{1}$ to judge device quality, these low-coherence regions should be avoided so that the incoherent-TLS-loss-limited median $T_{1}$ can be determined. Coupled with a spectral cleanliness metric, this variable describes the materials-induced decoherence of the device.

Large, correlated errors, such as those due to cosmic rays, can strongly affect superconducting qubit systems. Martinis (2020) Recent work has shown that cosmic rays can induce correlated, catastrophic qubit decoherence, with events occurring on average every 10 seconds. The effects of these events have been shown to last around 25 ms, with the most impactful of these events leading to $T_{1}<1\mathrm{\mu}s$ . McEwen et al. (2021) A combination of cosmic ray shielding Vepsäläinen et al. (2020); Cardani et al. (2020) and on-chip mitigation could solve this issue, but further investigation is required.

Little information is currently published on temporal cross-correlation or causal relations between performance metrics. This is an important area for future research.

In opposition to resonator performance, qubit $T_{1}$ increases as the sample decreases in temperature. (Lisenfeld et al., 2007, 2010) Thus, the confirmation of sample thermalization is necessary to ensure an optimal measurement environment.

The temperature variation of ambient-temperature electronics and microwave components, over time and even by time of day has been anecdotally shown to affect qubit performance. (Staley, ; Valery, Hill, and Poletto, ) The measurement of devices over long time periods can help avoid systematic, uncontrolled biasing of the measurement result.

V Cooldown-to-Cooldown Variation

The on-chip TLS configuration, including the frequencies and number of coherent TLS, has been observed to change with each cooldown, (Burnett et al., 2019) implying a "freeze out" effect where the population of coherent TLS in a device vary significantly between cooldowns but not within a single cooldown. What remains unclear is whether median time series data and performance metrics within a cooldown are consistent with subsequent cooldown data. Here, we define a subsequent cooldown to be cooling a sample to base temperature from ambient temperature without opening the dilution refrigerator.

The primary questions are: (1) Are the changes in the distributions of the performance metrics statistically significant between cooldowns? and (2) If so what are the magnitudes of these changes? Currently, the type and magnitude of these cooldown-to-cooldown changes of performance metrics are largely unstudied.

Across cooldowns, resonator $Q_{i}$ at high power has been shown to vary by more than 50 $\%$ , whereas TLS loss remains within the fit uncertainty. (Calusine et al., 2018) This is consistent with large cooldown-to-cooldown variations in experimental environment (which is a strong factor in high power $Q_{i}$ ) and negligible cooldown-to-cooldown variation in the incoherent TLS population.

VI Effects of Microwave Packaging

The microwave packaging that houses a superconducting circuit is generally transferable between systems and can thus be seen as independent of other setup-to-setup variations. This distinction becomes important when implementing interlaboratory comparisons of devices. Wirebonds, lossy materials, device thermalization, and box modes are all factors that can affect the perceived performance of superconducting quantum circuits. (Wenner et al., 2011; Rosenberg et al., 2019; Lienhard et al., 2019; Mergenthaler et al., 2020; Huang et al., 2021)

Wirebond inductance causes loss-inducing crosstalk that increases with length, whereas stray transmission can be suppressed by increasing the density of grounding wirebonds. (Wenner et al., 2011) A high density of short grounding wirebonds is commonly implemented to balance these conflicting effects, but more research is required to determine decoherence effects of wirebond repeatability in terms of variation in length and placement. Alternatives such as pogo pins (Bronn et al., 2018; Béjanin et al., 2016), air bridges, (Abuwasib, Krantz, and Delsing, 2013; Dunsworth et al., 2018; Chen et al., 2014) and bump bonds (Rosenberg et al., 2017; Foxen et al., 2018) can also be implemented in some instances and may allow greater experimental reproducibility.

When present, geometry-dependent box modes can couple to on-chip devices and cause circuit decoherence with nebulous behavior. Ideally, box modes should be far detuned from the spectrum of interest (Wenner et al., 2011; McConkey et al., 2018; Lienhard et al., 2019) in order to facilitate measurement reproducibility between sample boxes. A recent study suggests that package modes due to the Purcell effect can limit the qubit lifetime to $T_{1}\lesssim 384\mathrm{\mu}s$ . Huang et al. (2021) Box modes can also facilitate crosstalk between on-chip devices, leading to coherent errors and adding another layer of complexity to performance comparisons. 3D integration such as bump bonds and vias can allow greater control over the microwave environment of on-chip devices. (Rosenberg et al., 2019)

Sample boxes are often made from high purity Al or gold-plated Cu. (Lienhard et al., 2019) Cu boxes have strong thermalization properties but also generate loss due to normal metal conduction, although in some packaging designs this loss is orders of magnitude lower than other losses affecting qubits and resonators. (Lienhard et al., 2019) On the other hand, Al boxes are superconducting and thus do not introduce normal conducting loss, but are poor thermal conductors.

In addition, lossy materials such as dielectrics and normal metals are sometimes included in the vicinity of the chip, such as on circuit boards or as a paste to thermalize and adhere the sample to the box. These components are lossy and, in the case of thermal paste, are not necessarily reproducible to implement. The replacement of these components by constant or lower-loss substitutes, such as the implementation of superconducting circuit boards (Pappas et al., 2018) and clamps rather than paste, is likely to improve measurement repeatability with changing microwave packaging.

VII Setup-to-Setup Variation

Performance metric measurements of superconducting devices are heavily influenced by attenuation, filtering, amplifiers, wiring, and other components between the ambient-temperature electronics and the superconducting circuits. (Ranzani et al., 2013; Wang et al., 2021) Connections to ambient-temperature control electronics influence both qubit relaxation and pure dephasing. (Yeh et al., 2017; Yan et al., 2016; Ithier et al., 2005; Krinner et al., 2019) Noise properties can vary significantly even between sources from the same manufacturer. In addition, the magnitude of the passive infrared radiation load can vary between dilution refrigerator set-ups. Together, these factors add to the complexity of reproducing performance-enhancing results between systems.

In qubit frequency sweeps such as those in Ref. 21 (Fig. 6), distinct frequencies of consistent low coherence can be identified. These frequencies can be attributed to factors in the experimental chain such as control line modes and microwave carrier signal bleedthrough, and will therefore differ between setups. The identification and subsequent detuning of the qubit from these frequencies is critical to allow comparisons of materials-limited qubit coherence as well as to optimize qubit performance.

For resonator characterization, cryogenic calibration such as in Refs. 74 and 75 can account for the vast majority of the differences between experimental setups, save for those past the calibration plane which normally includes microwave packing and at least one connector. Additionally, hanger-mode resonator fitting is self-calibrating (Khalil et al., 2012; Probst et al., 2015; Gao, 2008) and thus changes in the microwave chain have little effect on the extracted $Q_{i}$ of hanger-mode resonators but a strong impact on reflection- and transmission-mode resonators.

Sample thermalization, a critical factor in measurement reproducibility, can be affected by experimental setup as well as sample packaging. The connection between the microwave packaging and base plate as well as thermalization between temperature stages determines in part the thermalization rate of the sample. Inefficient thermalization can lead to elevated sample temperatures and thus reduce qubit performance and elevate resonator performance. Thermalization of superconducting devices in dilution refrigerators has yet to be studied in depth, but a good thermal link and method for accurate sample temperature measurement would sufficiently address this concern. Possible thermometry methods include implementing the qubit itself as a thermometer.

VIII Cross-Wafer Variation and Variation Between Fabrication Runs

Nanofabrication processes for superconducting quantum devices are still largely performed in academic or small-scale cleanrooms that do not have the same process control or device reproducibility as large-scale semiconductor foundries. Industrial efforts in superconducting quantum device manufacturing have likely produced in-depth but unpublished coherence reproducibility studies. Thus, fabrication-induced coherence inhomogeneity across the wafer and between fabrication runs remain as significant unknowns. Variation in factors such as the superconducting transition temperatures, film thicknesses, etch rates, mechanical stress, and oxidation could have strong but currently unstudied effects on device performance.

As well, some of these small fabrication differences are believed to have a significant impact on TLS populations and therefore device performance. Indeed, resonators on the same chip have been shown to have $Q_{i}$ that vary by up to an order of magnitude. (Ohya et al., 2013)

As outlined in Ref. 23, on the order of 120 resonators are needed in order to extract individual losses from metal-substrate, substrate-air and metal-air interfaces as well as the substrate using the surface loss extraction technique. (Calusine et al., 2018) We posit that this can be thought of as an extreme upper bound for the number of resonators needed to capture performance fluctuations due to fabrication for a single device design and fabrication process.

Cross-wafer and fabrication-run variation in performance are especially difficult to characterize as these studies require taking into account errors in all other aspects of the experiment. This is the most nebulous of all the variation types seen in superconducting quantum circuit experiments. For this reason, ambient-temperature characterization methods that correlate to millikelvin microwave device performance are seen as a holy grail in superconducting quantum computing.

Table 1: Summary of error types that affect

Q_{i}

and

F\delta_{TLS}^{0}

measurements in planar superconducting microwave resonators, including error sources, typical error magnitudes, and possible solutions for minimizing or accounting for error. TBD: further investigation is required.

\sigma_{A}

: standard deviation of

A

\Delta A

: variation in A.

Type	Source	Magnitude	Solution
Measurement $\&$ analysis	$Q_{i}$ fit error	TBD	Design for low error regime
	$Q_{i}$ fit uncertainty	$\sigma_{Qi}\pm\sim 2\%$ (20 $\%$ ) for low (high) noise ¹¹1Typical values based on past measurements of Al hanger mode CPW resonators at the Boulder Cryogenic Quantum Testbed, University of Colorado Boulder.	Report uncertainty
	TLS fit uncertainty	$\sigma_{TLS}\sim\pm 5\%$ for power sweep (McRae et al., 2021)	Report uncertainty, reduce low power noise
Within one cooldown	Incoherent TLS	$\Delta Q_{i}\lesssim 30\%$ over several hours (Megrant et al., 2012; Neill et al., 2013)	Time average data
		$\sigma_{Qi}\sim$ 12 $\%$ over three hours (Moeed et al., 2019)
	Thermalization over time	$\Delta Q_{i}\lesssim$ orders of magnitude (Barends et al., 2011; McRae et al., 2020)	Ensure sample thermalization
			prior to measurement
Cooldown-to-cooldown	Microwave environment	$\Delta Q_{i,HP}>50\%$ (Calusine et al., 2018)	Report $\delta_{TLS}^{0}$ rather than low power $Q_{i}$
		$\Delta\delta_{TLS}^{0}\sim 0$ (Calusine et al., 2018)
Microwave packaging	Wirebonds	Varies	High density of low wirebonds, (Wenner et al., 2011)
			alternative solutions (Chen et al., 2014; Béjanin et al., 2016; Bronn et al., 2018; Abuwasib, Krantz, and Delsing, 2013; Dunsworth et al., 2018; Rosenberg et al., 2017; Foxen et al., 2018)
	Sample box material	Varies	TBD
	Box modes	Varies	Design to suppress box modes
	Lossy materials	Varies	Use lower-loss substitutes
Setup-to-setup	Microwave chain	Can lead to unphysical values in $Q_{i}$ (Wang et al., 2021)	Calibration or hanger-mode (Ranzani et al., 2013; Wang et al., 2021)
	Thermalization	Varies, can elevate $Q_{i}$	Good thermal link,
			$T$ measurement at sample
Cross-wafer	Resist, deposition,	Unknown	Measure a large number of
	etch inhomogeneity		nominally identical devices
Between fabrication runs	Changes in chamber	Unknown	Measure a large number of
	particulate, etch rates, etc.		nominally identical devices

Table 2: Summary of error types in

T_{1}

and

T_{\phi}

measurements of superconducting transmon qubits, including error sources, magnitudes, and possible solutions for minimizing or accounting for error. TBD: further investigation is required.

\sigma_{A}

: standard deviation of

A

\Delta A

: variation in A.

Type	Source	Magnitude	Solution
Measurement and analysis	Error in TLS detection	TBD	Automate detection process
Within one cooldown	Incoherent TLS	$\Delta T_{1}\sim 20\%$ (Klimov et al., 2018)	Time average data
	Coherent TLS	$\Delta T_{1}\lesssim$ order of magnitude (Klimov et al., 2018)	Measure over time and frequency Carroll et al. (2021)
	Cosmic rays	$\Delta T_{1}\lesssim$ order of magnitude (McEwen et al., 2021; Martinis, 2020)	Shielding, Vepsäläinen et al. (2020); Cardani et al. (2020) on-chip mitigation
	Thermalization over time	Varies (Lisenfeld et al., 2007, 2010)	Ensure sample thermalization
			prior to measurement
	Temperature of RT electronics	TBD	Periodic recalibration,
			temperature control
Cooldown-to-cooldown	Freeze-out of TLS	$\Delta T_{1}\lesssim$ order of magnitude (Klimov et al., 2018)	Detune from TLS
Microwave packaging	Wirebonds	Varies	High density of low wirebonds, (Wenner et al., 2011)
			alternative solutions (Bronn et al., 2018; Béjanin et al., 2016; Abuwasib, Krantz, and Delsing, 2013; Dunsworth et al., 2018; Chen et al., 2014; Rosenberg et al., 2017; Foxen et al., 2018)
	Sample box material	Varies	TBD
	Box modes	Varies	Design to suppress box modes
	Lossy materials	Varies	Use lower-loss substitutes
Setup-to-setup	Microwave chain	Very large (Krinner et al., 2019; Klimov et al., 2018)	Detune from regions
	Thermalization	Varies, can reduce $T_{1}$	Good thermal link,
			$T$ measurement at sample
Cross-wafer	Resist, deposition,	Unknown	TBD
	etch inhomogeneity
Between fabrication runs	Changes in chamber	Unknown	TBD
	particulate, etch rates, etc.

IX Reproducibility Guidelines

In this section, we provide a non-exhaustive summary of best practices as well as current estimates for orders of magnitude of variation of performance metrics for qubits and resonators (Tables 1 and 2). These guidelines could allow reproducibility and one-to-one comparisons in interlaboratory measurements, which could enable true breakthroughs in superconducting quantum device performance.

IX.1 Data guidelines:

•

In order to facilitate future data reanalysis based on new understandings, as well as one-to-one comparison, make data publicly available in common file formats such as csv or JSON and include sufficient metadata such as powers, temperatures, and number of averages as applicable.
•

Make data analysis routines version-controlled and publicly available, and state which version of analysis routines were used.
•

Provide order-of-magnitude error estimates, even if all error sources were not explicitly probed. Include fit uncertainties.

IX.2 Experimental guidelines:

•

Interleave measurements of different performance metrics when possible to enable cross-correlation analysis.
•

Record laboratory and control electronics temperature to enable cross-correlation analysis with the performance metrics.
•

Design packaging to suppress box modes and substitute lossy materials near the sample to allow measurement repeatability between microwave packages.
•

For resonator characterization, hanger-mode geometry should be implemented, or cryogenic calibration measurements should be performed to account for the effects of the microwave chain between the vector network analyzer and the sample.
•

Characterize qubit performance over a time span of at least 1-2 days and a small frequency bandwidth around the idle frequency of the qubit in order to capture coherent TLS effects.
•

For a more complete qubit characterization, include a spectral cleanliness metric along with median $T_{1}$ and $T_{\phi}$ at an incoherent-TLS-loss-limited frequency or at the idle frequency of the qubit.
•

Measure more than one nominally identical device when possible in order to capture some element of fabrication-induced performance variation.
•

Perform multiple cooldowns when possible in order to capture changes in coherent TLS population when cycling to room temperature.

IX.3 Reporting guidelines:

•

Include description of entire experimental set-up, including microwave wiring, sample mount, and microwave packaging.
•

Include description of device design, including material filling factors when possible.
•

Report median and an inter-percentile as summary statistics of the times series data. These are robust statistics of central tendency and spread, unlike the mean and standard deviation.
•

Include accurate sample temperature data when possible.

X Expanding the Frontier of Reproducible Coherence Characterization

In previous sections, we outline known sources of error and variation in superconducting quantum device measurement, and we recommend best practices for reproducible coherence characterization of these devices. In this section, we highlight areas for knowledge expansion and propose experiments to heighten our understanding in these areas.

One important goal is to increase understanding of loss sources, fluctuations of loss, and magnitudes of loss in superconducting devices. Measuring both longer- and shorter-scale time series of resonator $Q_{i}$ and correlating them with qubit $T_{1}$ could provide this additional information. Particularly, longer measurements would allow the investigation of fluctuations in performance due to changes in ambient temperature and coherent TLS frequency drift. In a similar vein, the development of a rapid TLS measurement protocol would allow investigations of TLS loss fluctuations over time.

The comparison of fluctuations of low-power and high-power $Q_{i}$ could allow us to more easily distinguish between types of loss.

It is critical understand the freeze-out effect in both incoherent and coherent TLS populations. Such effects can be studied by cooldown-to-cooldown variation of both qubits and resonators, where devices are cycled from ambient to millikelvin temperatures many times. Of course, the reported data would conform to the guidelines laid out in Section IX.

Accurate temperature measurements of the sample itself rather than the mixing chamber stage would allow investigations of thermalization, both of the sample to the microwave packaging and the package to the base plate.

A comprehensive study of microwave packing, including geometry, bulk material, circuit board, and thermalization method, must be performed in order to understand the full extent of these effects.

The investigation that relies on reproducible coherence characterization in all other aspects is the investigation of fabrication method effects. This set of experiments will allow us to optimize superconducting quantum device performance and could possibly enable large-scale superconducting quantum computing in the future. An attractive first step would be the implementation of rapid resonator characterization to allow the collection of larger datasets.

Finally, multiqubit benchmarking, while not included in this work, is essential to the investigation of correlated and spatially-dependent performance fluctuation sources such quasiparticles due to cosmic rays, Martinis (2020); McEwen et al. (2021) and further investigations into spatially-dependent performance correlations is warranted.

XI Conclusion

In conclusion, we have explored error contributions to superconducting device performance metrics from various experimental areas, including measurement and analysis error, fluctuations over time, cooldown-to-cooldown variations, effects of microwave packaging, setup-to-setup variations, and fabrication-induced variations. Experimental areas requiring further study were elaborated upon, and experimental guidelines for reducing and accounting for these errors were noted.

This article is not intended to cast aspersions on the reported results in the literature. Steady progress in superconducting quantum device performance has been observed for decades in academic and industrial labs all over the world, in contrast to other qubit modalities. (Frolov, 2021) Large improvements in device performance, especially those due to design modification, have been largely reproducible from one laboratory to another. That is not to say that all improvements are comparable or even reproducible. As the field matures and reported improvements become more modest, it is difficult to be certain the results are reproducible.

Looking to the future of the field, there are a number of paths towards reproducible coherence characterization. The simplest path is to report experimental details regarding known variation sources, which would enable interlaboratory performance improvements. On the more ambitious end is the development of a standard, optimized, open-source dilution refrigerator for coherence characterization (including standardized wiring, control electronics, mounts, data acquisition and analysis software, and commissioning and verification procedures). Such standard experimental set-up could then be straightforwardly reproduced.

Acknowledgements.

The authors would like to thank Clemens Müller, Stefano Poletto, Alexa Staley, Michael Vissers, and Ruichen Zhao for helpful discussions and feedback on the manuscript. This material is based upon work supported by the U.S. Department of Energy, Office of Science, National Quantum Information Science Research Centers, Superconducting Quantum Materials and Systems Center (SQMS) under the contract No. DE-AC02-07CH11359.

Data Availability

The data that support the findings of this study are openly available in Boulder-Cryogenic-Quantum-Testbed/data at https://doi.org/10.5281/zenodo.4902377.

References

Gambetta, Chow, and Steffen (2017) J. M. Gambetta, J. M. Chow, and M. Steffen, “Building logical qubits in a superconducting quantum computing system,” npj Quantum Information 3, 2 (2017).
Arute et al. (2019) F. Arute, K. Arya, R. Babbush, D. Bacon, J. C. Bardin, R. Barends, R. Biswas, S. Boixo, F. G. S. L. Brandao, D. A. Buell, B. Burkett, Y. Chen, Z. Chen, B. Chiaro, R. Collins, W. Courtney, A. Dunsworth, E. Farhi, B. Foxen, A. Fowler, C. Gidney, M. Giustina, R. Graff, K. Guerin, S. Habegger, M. P. Harrigan, M. J. Hartmann, A. Ho, M. Hoffmann, T. Huang, T. S. Humble, S. V. Isakov, E. Jeffrey, Z. Jiang, D. Kafri, K. Kechedzhi, J. Kelly, P. V. Klimov, S. Knysh, A. Korotkov, F. Kostritsa, D. Landhuis, M. Lindmark, E. Lucero, D. Lyakh, S. Mandrà, J. R. McClean, M. McEwen, A. Megrant, X. Mi, K. Michielsen, M. Mohseni, J. Mutus, O. Naaman, M. Neeley, C. Neill, M. Y. Niu, E. Ostby, A. Petukhov, J. C. Platt, C. Quintana, E. G. Rieffel, P. Roushan, N. C. Rubin, D. Sank, K. J. Satzinger, V. Smelyanskiy, K. J. Sung, M. D. Trevithick, A. Vainsencher, B. Villalonga, T. White, Z. J. Yao, P. Yeh, A. Zalcman, H. Neven, and J. M. Martinis, “Quantum supremacy using a programmable superconducting processor,” Nature 574, 505–510 (2019).
Müller, Cole, and Lisenfeld (2019) C. Müller, J. H. Cole, and J. Lisenfeld, “Towards understanding two-level-systems in amorphous solids: insights from quantum circuits,” Rep. Prog. Phys. 82, 124501 (2019).
McRae et al. (2020) C. R. H. McRae, H. Wang, J. Gao, M. Vissers, T. Brecht, A. Dunsworth, D. Pappas, and J. Mutus, “Materials loss measurements using superconducting microwave resonators,” 091101 (2020), 10.1063/5.0017378, 2006.04718 .
Wang et al. (2009) H. Wang, M. Hofheinz, J. Wenner, M. Ansmann, R. Bialczak, M. Lenander, E. Lucero, M. Neeley, A. O’Connell, D. Sank, et al., “Improving the coherence time of superconducting coplanar resonators,” Appl. Phys. Lett. 95, 233508 (2009).
Macha et al. (2010) P. Macha, S. van Der Ploeg, G. Oelsner, E. Il’ichev, H.-G. Meyer, S. Wünsch, and M. Siegel, “Losses in coplanar waveguide resonators at millikelvin temperatures,” Appl. Phys. Lett. 96, 062503 (2010).
Sage et al. (2011) J. M. Sage, V. Bolkhovsky, W. D. Oliver, B. Turek, and P. B. Welander, “Study of loss in superconducting coplanar waveguide resonators,” J. Appl. Phys. 109, 063915 (2011).
Megrant et al. (2012) A. Megrant, C. Neill, R. Barends, B. Chiaro, Y. Chen, L. Feigl, J. Kelly, E. Lucero, M. Mariantoni, P. J. O’Malley, et al., “Planar superconducting resonators with internal quality factors above one million,” Appl. Phys. Lett. 100, 113510 (2012).
Richardson et al. (2016) C. Richardson, N. Siwak, J. Hackley, Z. Keane, J. Robinson, B. Arey, I. Arslan, and B. Palmer, “Fabrication artifacts and parallel loss channels in metamorphic epitaxial aluminum superconducting resonators,” Supercond. Sci. Technol. 29, 064003 (2016).
Earnest et al. (2018) C. T. Earnest, J. H. Béjanin, T. G. McConkey, E. A. Peters, A. Korinek, H. Yuan, and M. Mariantoni, “Substrate surface engineering for high-quality silicon/aluminum superconducting resonators,” Supercond. Sci. Technol. 31, 125013 (2018).
Vissers et al. (2010) M. R. Vissers, J. Gao, D. S. Wisbey, D. A. Hite, C. C. Tsuei, A. D. Corcoles, M. Steffen, and D. P. Pappas, “Low loss superconducting titanium nitride coplanar waveguide resonators,” Appl. Phys. Lett. 97, 232509 (2010).
Ohya et al. (2013) S. Ohya, B. Chiaro, A. Megrant, C. Neill, R. Barends, Y. Chen, J. Kelly, D. Low, J. Mutus, P. O’Malley, et al., “Room temperature deposition of sputtered tin films for superconducting coplanar waveguide resonators,” Supercond. Sci. Technol. 27, 015009 (2013).
Calusine et al. (2018) G. Calusine, A. Melville, W. Woods, R. Das, C. Stull, V. Bolkhovsky, D. Braje, D. Hover, D. K. Kim, X. Miloshi, et al., “Analysis and mitigation of interface losses in trenched superconducting coplanar waveguide resonators,” Appl. Phys. Lett. 112, 062601 (2018).
Lock et al. (2019) E. H. Lock, P. Xu, T. Kohler, L. Camacho, J. Prestigiacomo, Y. J. Rosen, and K. D. Osborn, “Using surface engineering to modulate superconducting coplanar microwave resonator performance,” IEEE Trans. Appl. Supercond. 29, 1–8 (2019).
Gao et al. (2008) J. Gao, M. Daal, A. Vayonakis, S. Kumar, J. Zmuidzinas, B. Sadoulet, B. A. Mazin, P. K. Day, and H. G. Leduc, “Experimental evidence for a surface distribution of two-level systems in superconducting lithographed microwave resonators,” Appl. Phys. Lett. 92, 152505 (2008).
Kumar et al. (2008) S. Kumar, J. Gao, J. Zmuidzinas, B. A. Mazin, H. G. LeDuc, and P. K. Day, “Temperature dependence of the frequency and noise of superconducting coplanar waveguide resonators,” Appl. Phys. Lett. 92, 123503 (2008).
Wisbey et al. (2010) D. S. Wisbey, J. Gao, M. R. Vissers, F. C. da Silva, J. S. Kline, L. Vale, and D. P. Pappas, “Effect of metal/substrate interfaces on radio-frequency loss in superconducting coplanar waveguides,” J. Appl. Phys. 108, 093918 (2010).
Goetz et al. (2016) J. Goetz, F. Deppe, M. Haeberlein, F. Wulschner, C. W. Zollitsch, S. Meier, M. Fischer, P. Eder, E. Xie, K. G. Fedorov, et al., “Loss mechanisms in superconducting thin film microwave resonators,” J. Appl. Phys. 119, 015304 (2016).
Burnett et al. (2019) J. J. Burnett, A. Bengtsson, M. Scigliuzzo, D. Niepce, M. Kudra, P. Delsing, and J. Bylander, “Decoherence benchmarking of superconducting qubits,” npj Quantum Information 5, 1–8 (2019), 1901.04417 .
Schlör et al. (2019) S. Schlör, J. Lisenfeld, C. Müller, A. Bilmes, A. Schneider, D. P. Pappas, A. V. Ustinov, and M. Weides, “Correlating Decoherence in Transmon Qubits: Low Frequency Noise by Single Fluctuators,” 123, 190502 (2019), 1901.05352 .
Klimov et al. (2018) P. V. Klimov, J. Kelly, Z. Chen, M. Neeley, A. Megrant, B. Burkett, R. Barends, K. Arya, B. Chiaro, Y. Chen, A. Dunsworth, A. Fowler, B. Foxen, C. Gidney, M. Giustina, R. Graff, T. Huang, E. Jeffrey, E. Lucero, J. Y. Mutus, O. Naaman, C. Neill, C. Quintana, P. Roushan, D. Sank, A. Vainsencher, J. Wenner, T. C. White, S. Boixo, R. Babbush, V. N. Smelyanskiy, H. Neven, and J. M. Martinis, “Fluctuations of Energy-Relaxation Times in Superconducting Qubits,” Physical Review Letters 121 (2018), 10.1103/PhysRevLett.121.090502.
McRae et al. (2021) C. R. H. McRae, A. McFadden, R. Zhao, H. Wang, J. Long, T. Zhao, S. Park, M. Bal, C. J. Palmstrom, and D. P. Pappas, “Cryogenic microwave loss in epitaxial Al/GaAs/Al trilayers for superconducting circuits,” 129 (2021), 10.1063/5.0029855.
Woods et al. (2019) W. Woods, G. Calusine, A. Melville, A. Sevi, E. Golden, D. K. Kim, D. Rosenberg, J. L. Yoder, and W. D. Oliver, “Determining interface dielectric losses in superconducting coplanar-waveguide resonators,” Phys. Rev. Applied 12, 014012 (2019).
Krantz et al. (2019) P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “A quantum engineer’s guide to superconducting qubits,” Applied Physics Reviews, Applied Physics Reviews 6, 021318 (2019).
Gao et al. (2021) Y. Y. Gao, M. A. Rol, S. Touzard, and C. Wang, “A practical guide for building superconducting quantum devices,” arXiv:2106.06173 [quant-ph] (2021), arXiv: 2106.06173.
Zmuidzinas (2012) J. Zmuidzinas, “Superconducting microresonators: Physics and applications,” Annu. Rev. Condens. Matter Phys. 3, 169–214 (2012).
De Graaf et al. (2018) S. De Graaf, L. Faoro, J. Burnett, A. Adamyan, A. Y. Tzalenchuk, S. Kubatkin, T. Lindström, and A. Danilov, “Suppression of low-frequency charge noise in superconducting resonators by surface spin desorption,” Nat. Commun. 9, 1–6 (2018).
Gao (2008) J. Gao, The physics of superconducting microwave resonators, Ph.D. thesis, California Institute of Technology (2008).
Khalil et al. (2012) M. Khalil, M. Stoutimore, F. Wellstood, and K. Osborn, “An analysis method for asymmetric resonator transmission applied to superconducting devices,” J. Appl. Phys. 111, 054510 (2012).
Kimmel, Low, and Yoder (2015) S. Kimmel, G. H. Low, and T. J. Yoder, “Robust calibration of a universal single-qubit gate set via robust phase estimation,” Phys. Rev. A 92, 062315 (2015).
Barends et al. (2013) R. Barends, J. Kelly, A. Megrant, D. Sank, E. Jeffrey, Y. Chen, Y. Yin, B. Chiaro, J. Mutus, C. Neill, et al., “Coherent josephson qubit suitable for scalable quantum integrated circuits,” Physical review letters 111, 080502 (2013).
(32) A. N. Staley, personal communication.
(33) J. Valery, A. Hill, and S. Poletto, personal communication.
Pappas et al. (2011) D. P. Pappas, M. R. Vissers, D. S. Wisbey, J. S. Kline, and J. Gao, “Two level system loss in superconducting microwave resonators,” IEEE Trans. Appl. Supercond. 21, 871–874 (2011).
Serniak et al. (2018) K. Serniak, M. Hays, G. de Lange, S. Diamond, S. Shankar, L. Burkhart, L. Frunzio, M. Houzet, and M. Devoret, “Hot nonequilibrium quasiparticles in transmon qubits,” Phys. Rev. Lett. 121, 157701 (2018).
Song et al. (2009) C. Song, T. W. Heitmann, M. P. DeFeo, K. Yu, R. McDermott, M. Neeley, J. M. Martinis, and B. L. Plourde, “Microwave response of vortices in superconducting thin films of Re and Al,” Phys. Rev. B 79, 174512 (2009).
Draščić (2016) B. Draščić, “Sampling reconstruction of stochastic signals– the roots in the fifties,” Austrian Journal of Statistics 36, 65–72 (2016).
Kay (1988) S. M. Kay, Modern spectral estimation: theory and application (Pearson Education India, 1988).
Pearl (2009) J. Pearl, Causality (Cambridge university press, 2009).
Heckert et al. (2002) N. A. Heckert, J. J. Filliben, C. M. Croarkin, B. Hembree, W. F. Guthrie, P. Tobias, and J. Prinz, “Nist/sematech e-handbook of statistical methods,” (2002).
Box et al. (2005) G. E. Box, W. H. Hunter, S. Hunter, et al., Statistics for Experimenters: Design, Innovation, and Discovery (Wiley-Interscience; 2nd edition, 2005).
Guerrier, Molinari, and Stebler (2016) S. Guerrier, R. Molinari, and Y. Stebler, “Theoretical limitations of allan variance-based regression for time series model estimation,” IEEE Signal Processing Letters 23, 597–601 (2016).
Siraya (2020) T. N. Siraya, “Statistical interpretation of the allan variance as a characteristic of measuring and navigation devices,” Gyroscopy and Navigation 11, 105–114 (2020).
Moeed et al. (2019) M. S. Moeed, C. T. Earnest, J. H. Béjanin, A. S. Sharafeldin, and M. Mariantoni, “Improving the time stability of superconducting planar resonators,” MRS Advances 4, 2201–2215 (2019).
Costa and Shrapnel (2016) F. Costa and S. Shrapnel, “Quantum causal modelling,” New Journal of Physics 18, 063032 (2016).
Milz and Modi (2021) S. Milz and K. Modi, “Quantum stochastic processes and quantum non-markovian phenomena,” PRX Quantum 2, 030201 (2021).
White et al. (2021) G. A. L. White, F. A. Pollock, L. C. L. Hollenberg, C. D. Hill, and K. Modi, “Diagnosing temporal quantum correlations: compressed non-Markovian calipers,” arXiv:2107.13934 [quant-ph] (2021), arXiv: 2107.13934.
Simmonds et al. (2004) R. W. Simmonds, K. M. Lang, D. A. Hite, S. Nam, D. P. Pappas, and J. M. Martinis, “Decoherence in josephson phase qubits from junction resonators,” Phys. Rev. Lett. 93, 077003 (2004).
Müller et al. (2015) C. Müller, J. Lisenfeld, A. Shnirman, and S. Poletto, “Interacting two-level defects as sources of fluctuating high-frequency noise in superconducting circuits,” Physical Review B 92 (2015), 1503.01637 .
Béjanin et al. (2021) J. H. Béjanin, C. T. Earnest, A. S. Sharafeldin, and M. Mariantoni, “Interacting Defects Generate Stochastic Fluctuations in Superconducting Qubits,” , 1–13 (2021), arXiv:2106.15748 .
Carroll et al. (2021) M. Carroll, S. Rosenblatt, P. Jurcevic, I. Lauer, and A. Kandala, “Dynamics of superconducting qubit relaxation times,” , 1–10 (2021), arXiv:2105.15201 .
Tersoff and Hannon (2021) J. Tersoff and J. Hannon, “Modeling of qubit coherence variability due to two-level systems,” (2021).
(53) IBM, personal communication.
Martinis (2020) J. M. Martinis, “Saving superconducting quantum processors from qubit decay and correlated errors generated by gamma and cosmic rays,” arXiv , 1–9 (2020), arXiv:2012.06137 .
McEwen et al. (2021) M. McEwen, L. Faoro, K. Arya, A. Dunsworth, T. Huang, S. Kim, B. Burkett, A. Fowler, F. Arute, J. C. Bardin, A. Bengtsson, A. Bilmes, B. B. Buckley, N. Bushnell, Z. Chen, R. Collins, S. Demura, A. R. Derk, C. Erickson, M. Giustina, S. D. Harrington, S. Hong, E. Jeffrey, J. Kelly, P. V. Klimov, F. Kostritsa, P. Laptev, A. Locharla, X. Mi, K. C. Miao, S. Montazeri, J. Mutus, O. Naaman, M. Neeley, C. Neill, A. Opremcak, C. Quintana, N. Redd, P. Roushan, D. Sank, K. J. Satzinger, V. Shvarts, T. White, Z. J. Yao, P. Yeh, J. Yoo, Y. Chen, V. Smelyanskiy, J. M. Martinis, H. Neven, A. Megrant, L. Ioffe, and R. Barends, “Resolving catastrophic error bursts from cosmic rays in large arrays of superconducting qubits,” (2021), arXiv:2104.05219 .
Vepsäläinen et al. (2020) A. P. Vepsäläinen, A. H. Karamlou, J. L. Orrell, A. S. Dogra, B. Loer, F. Vasconcelos, D. K. Kim, A. J. Melville, B. M. Niedzielski, J. L. Yoder, S. Gustavsson, J. A. Formaggio, B. A. VanDevender, and W. D. Oliver, “Impact of ionizing radiation on superconducting qubit coherence,” Nature 584, 551–556 (2020), arXiv:2001.09190 .
Cardani et al. (2020) L. Cardani, F. Valenti, N. Casali, G. Catelani, T. Charpentier, M. Clemenza, I. Colantoni, A. Cruciani, L. Gironi, L. Grünhaupt, et al., “Reducing the impact of radioactivity on quantum circuits in a deep-underground facility,” arXiv preprint arXiv:2005.02286 (2020).
Lisenfeld et al. (2007) J. Lisenfeld, A. Lukashenko, M. Ansmann, J. M. Martinis, and A. V. Ustinov, “Temperature dependence of coherent oscillations in Josephson phase qubits,” Physical Review Letters 99, 1–4 (2007), 0705.0095 .
Lisenfeld et al. (2010) J. Lisenfeld, C. Müller, J. H. Cole, P. Bushev, A. Lukashenko, A. Shnirman, and A. V. Ustinov, “Measuring the temperature dependence of individual two-level systems by direct coherent control,” Physical Review Letters 105, 3–6 (2010), 1008.0337 .
Wenner et al. (2011) J. Wenner, M. Neeley, R. C. Bialczak, M. Lenander, E. Lucero, A. D. O’Connell, D. Sank, H. Wang, M. Weides, A. N. Cleland, et al., “Wirebond crosstalk and cavity modes in large chip mounts for superconducting qubits,” Supercond. Sci. Technol. 24, 065001 (2011).
Rosenberg et al. (2019) D. Rosenberg, S. Weber, D. Conway, D. Yost, J. Mallek, G. Calusine, R. Das, D. Kim, M. Schwartz, W. Woods, et al., “3d integration and packaging for solid-state qubits,” arXiv preprint arXiv:1906.11146 (2019).
Lienhard et al. (2019) B. Lienhard, J. Braumüller, W. Woods, D. Rosenberg, G. Calusine, S. Weber, A. Vepsäläinen, K. O’Brien, T. P. Orlando, S. Gustavsson, et al., “Microwave packaging for superconducting qubits,” arXiv preprint arXiv:1906.05425 (2019).
Mergenthaler et al. (2020) M. Mergenthaler, S. Paredes, P. Müller, C. Müller, S. Filipp, M. Sandberg, J. Hertzberg, V. P. Adiga, M. Brink, and A. Fuhrer, “Ultrahigh Vacuum Packaging and Surface Cleaning for Quantum Devices,” , 1–6 (2020), arXiv:2010.12090 .
Huang et al. (2021) S. Huang, B. Lienhard, G. Calusine, A. Vepsäläinen, J. Braumüller, D. K. Kim, A. J. Melville, B. M. Niedzielski, J. L. Yoder, B. Kannan, T. P. Orlando, S. Gustavsson, and W. D. Oliver, “Microwave Package Design for Superconducting Quantum Processors,” PRX Quantum 2, 1–15 (2021), arXiv:2012.01438 .
Bronn et al. (2018) N. T. Bronn, V. P. Adiga, S. B. Olivadese, X. Wu, J. M. Chow, and D. P. Pappas, “High coherence plane breaking packaging for superconducting qubits,” Quantum science and technology 3, 024007 (2018).
Béjanin et al. (2016) J. Béjanin, T. McConkey, J. Rinehart, C. Earnest, C. McRae, D. Shiri, J. Bateman, Y. Rohanizadegan, B. Penava, P. Breul, et al., “Three-dimensional wiring for extensible quantum computing: The quantum socket,” Phys. Rev. Applied 6, 044010 (2016).
Abuwasib, Krantz, and Delsing (2013) M. Abuwasib, P. Krantz, and P. Delsing, “Fabrication of large dimension aluminum air-bridges for superconducting quantum circuits,” J. Vac. Sci. Technol. B 31, 3 (2013).
Dunsworth et al. (2018) A. Dunsworth, R. Barends, Y. Chen, Z. Chen, B. Chiaro, A. Fowler, B. Foxen, E. Jeffrey, J. Kelly, P. Klimov, et al., “A method for building low loss multi-layer wiring for superconducting microwave devices,” Appl. Phys. Lett. 112, 063502 (2018).
Chen et al. (2014) Z. Chen, A. Megrant, J. Kelly, R. Barends, J. Bochmann, Y. Chen, B. Chiaro, A. Dunsworth, E. Jeffrey, J. Mutus, et al., “Fabrication and characterization of aluminum airbridges for superconducting microwave circuits,” Appl. Phys. Lett. 104, 052602 (2014).
Rosenberg et al. (2017) D. Rosenberg, D. Kim, R. Das, D. Yost, S. Gustavsson, D. Hover, P. Krantz, A. Melville, L. Racz, G. O. Samach, S. J. Weber, F. Yan, J. Yoder, A. J. Kerman, and W. D. Oliver, “3D integrated superconducting qubits,” npj Quantum Inf. 3 (2017).
Foxen et al. (2018) B. Foxen, J. Y. Mutus, E. Lucero, R. Graff, A. Megrant, Y. Chen, C. Quintana, B. Burkett, J. Kelly, E. Jeffrey, Y. Yang, A. Yu, K. Arya, R. Barends, Z. Chen, B. Chiaro, A. Dunsworth, A. Fowler, C. Gidney, M. Giustina, T. Huang, P. Klimov, M. Neeley, C. Neill, P. Roushan, D. Sank, A. Vainsencher, J. Wenner, T. C. White, and J. M. Martinis, “Qubit compatible superconducting interconnects,” Quant. Sci. Tech. 3, 014005 (2018).
McConkey et al. (2018) T. McConkey, J. Béjanin, C. Earnest, C. McRae, Z. Pagel, J. Rinehart, and M. Mariantoni, “Mitigating leakage errors due to cavity modes in a superconducting quantum computer,” Quantum Science and Technology 3, 034004 (2018).
Pappas et al. (2018) D. P. Pappas, D. E. David, R. E. Lake, M. Bal, R. B. Goldfarb, D. A. Hite, E. Kim, H. S. Ku, C. R. H. Mcrae, L. D. Pappas, A. Roshko, J. G. Wen, B. L. T. Plourde, I. Arslan, and X. Wu, “Enhanced Superconducting Transition Temperature in Electroplated Rhenium,” , 1–8 (2018).
Ranzani et al. (2013) L. Ranzani, L. Spietz, Z. Popovic, and J. Aumentado, “Two-port microwave calibration at millikelvin temperatures,” Rev. Sci. Instrum. 84, 034704 (2013).
Wang et al. (2021) H. Wang, S. Singh, C. R. H. McRae, J. C. Bardin, S. X. Lin, N. Messaoudi, A. R. Castelli, Y. J. Rosen, E. T. Holland, D. P. Pappas, and J. Y. Mutus, “Cryogenic single-port calibration for superconducting microwave resonator measurements,” (2021), arXiv:2103.01491 .
Yeh et al. (2017) J.-H. Yeh, J. LeFebvre, S. Premaratne, F. Wellstood, and B. Palmer, “Microwave attenuators for use with quantum devices below 100 mk,” J. Appl. Phys. 121, 224501 (2017).
Yan et al. (2016) F. Yan, S. Gustavsson, A. Kamal, J. Birenbaum, A. P. Sears, D. Hover, T. J. Gudmundsen, D. Rosenberg, G. Samach, S. Weber, J. L. Yoder, T. P. Orlando, J. Clarke, A. J. Kerman, and W. D. Oliver, “The flux qubit revisited to enhance coherence and reproducibility,” Nature Communications 7, 1–9 (2016), 1508.06299 .
Ithier et al. (2005) G. Ithier, E. Collin, P. Joyez, P. J. Meeson, D. Vion, D. Esteve, F. Chiarello, A. Shnirman, Y. Makhlin, J. Schriefl, and G. Schön, “Decoherence in a superconducting quantum bit circuit,” Physical Review B - Condensed Matter and Materials Physics 72, 1–22 (2005).
Krinner et al. (2019) S. Krinner, S. Storz, P. Kurpiers, P. Magnard, J. Heinsoo, R. Keller, J. Lütolf, C. Eichler, and A. Wallraff, “Engineering cryogenic setups for 100-qubit scale superconducting circuit systems,” EPJ Quantum Technology 6 (2019), 10.1140/epjqt/s40507-019-0072-0, 1806.07862 .
Probst et al. (2015) S. Probst, F. Song, P. Bushev, A. Ustinov, and M. Weides, “Efficient and robust analysis of complex scattering data under noise in microwave resonators,” Rev. Sci. Instrum. 86, 024706 (2015).
Neill et al. (2013) C. Neill, A. Megrant, R. Barends, Y. Chen, B. Chiaro, J. Kelly, J. Mutus, P. O’Malley, D. Sank, J. Wenner, et al., “Fluctuations from edge defects in superconducting resonators,” Applied Physics Letters 103, 072601 (2013).
Barends et al. (2011) R. Barends, J. Wenner, M. Lenander, Y. Chen, R. C. Bialczak, J. Kelly, E. Lucero, P. O’Malley, M. Mariantoni, D. Sank, et al., “Minimizing quasiparticle generation from stray infrared light in superconducting quantum circuits,” Appl. Phys. Lett. 99, 113507 (2011).
Frolov (2021) S. Frolov, “Quantum computing’s reproducibility crisis: Majorana fermions,” Nature 592, 350–352 (2021).