Scalable Nanophotonic-Electronic Spiking Neural Networks

Excitability describes the ability of a system to quickly and temporarily deviate from its quiescent state following small perturbations and can be rigorously described through bifurcation analysis as done by Izhikevich [15]. Biological neurons are dynamical systems and have been classified into saddle-node and Andronov-Hopf bifurcations which correspond to integrator and resonator neurons respectively. Simply put, integrator neurons integrate their inputs and will generate a spike upon reaching some dynamic threshold, while a resonator neuron undergoes some internal subthreshold oscillation with an increased response and likelihood to generate a spike for inputs that fall at specific phases of a resonant frequency.

Computationally useful spiking neurons, however, need not be entirely biologically plausible. Instead, behavior is commonly summarized by the leaky-integrate-and-fire (LIF) neuron model. In the LIF model, the membrane potential constantly undergoes exponential decay towards its resting potential with discrete jumps at each input spike. When the membrane potential reaches a fixed threshold, the spike is generated and the potential is instantaneously returned to a reset potential. LIF neurons are only able to represent integrator neurons and lose much of the complexity of behaviors seen in biological neurons. Alternatively, Izhikevich devised a neuron model which faithfully reproduces a wide range of biologically observed behaviors using only four parameters and two coupled differential equations [16]. For a brief summary of computationally relevant neuron behaviors and a comparison of neuron models see [17]. Other taxonomies exist to classify neuron types according to these behaviors, though some evidence has shown that biological neurons may flexibly switch between these types based on the history of the cell [18]. As such, an ideal hardware implementation of spiking neurons would be capable of representing a range of neuron types for maximal computational ability.

A number of semiconductor lasers have been explored which create isomorphisms between the time dynamics of material parameters of active photonic elements and the cellular mechanisms of biological neurons. Researchers have exploited the time dynamics of photocarriers, thermal diffusion, optical modes, and polarization competition to create excitable laser devices with varying degrees of faithfulness to the biology. Photonic spiking neurons can be most meaningfully divided into two categories based on whether the device can accept optical or electrical inputs—some devices can be modulated by either, but electrical input may be preferred for the advantages in system design discussed in Sec. II-B.

Optical devices can be further classified into coherent and incoherent devices based on how incoming wavelengths are used to excite the active medium. In coherent excitable semiconductor lasers [19, 20, 21, 22, 23], the incoming signal interacts with a lasing cavity mode on the same wavelength to modulate the output signal directly. Excitability is induced by disturbing the balance between competing modes or polarizations which, with sufficient input energy, temporarily drive the extinction of one mode and amplification of the other. Bandwidth for such devices is bound by the cavity Q factor, with a time constant for energy dissipation given by $\tau=Q/\omega_{0}$. For incoherent devices [24, 25, 26, 27] the incoming signal interacts with some element within the cavity that indirectly modulates the output signal. This may take the form of optical pumping of the laser medium, or otherwise modulating the carrier populations which affect gain and saturation properties. Bandwidth for such approaches are limited by the dynamics of these carrier populations which are material dependent. Alternatively, optoelectronic approaches [28, 29] can allow for the design of analog circuitry with time-dynamics that can be fit to a variety of available neuron models, with lasers modulated by current injection in response to processed photodetector input. Optoelectronic designs are mainly limited by the total bandwidth of integrated photodetectors and electronics, though some estimates suggest that bandwidths upwards of 10 GHz can be expected; see [30] for a more in-depth review of various excitable semiconductor lasers with discussions of bifurcation paralleling Izhikevich’s analysis.

Given the ability of silicon waveguides to simultaneously support a wide range of wavelengths with negligible loss, on-chip optical networks are most efficiently parallelized using wavelength division multiplexing (WDM). Time-division multiplexing (TDM) offers another scheme for sharing computing resources over time, but the asynchronous and stochastic nature of SNNs is not likely to benefit from this technique. Using WDM, signals from each neuron can be routed according to wavelength and resources for matrix multiplication may potentially be used for multiple independent operations to support weight sharing and convolution. To support such architectures, different neurons must be distinguishable by output wavelength. However, the system does not need a unique wavelength for each neuron since most SNN architectures group neurons into layers that provide an additional level of hierarchy for routing structures.

Using a WDM approach, arrayed waveguide grating routers (AWGR) can be used to support all-to-all routing schemes between neural layers [31, 32, 33]. Inputs to each layer would be passed through reconfigurable optical matrix multipliers such as cross-bar networks, micro-ring resonator (MRR) banks, and mach-zehnder interferometry (MZI) meshes. MZI meshes can perform unitary matrix transformations that correspond to lossless multiplication and are thus particularly suitable for low-power neuromorphic computing. See [34] for a longer discussion on the design trade-offs between each of these devices. Sec. III-B describes our MZI mesh architecture, while Sec.III-C details algorithms for training SNNs using MZI meshes.

Neurons provide nonlinearity and signal regeneration between each neural network layer. Our previous work [35] presents an optoelectronic neuron design with projected energy efficiency on the order of $200\,aJ/\text{spike}$. Because the time-scales of electrical circuits are more tunable than photonic nonlinear materials, the neuron is more easily programmable while still taking advantage of low-loss communication provided by photonic interconnects. This design closely matches the behavioral characteristics of the Izhikevich neuron model to achieve a variety of neural behaviors. To move a step forward in realizing attojoule energy efficiencies, we have updated this design on a more advanced foundry platform.

Our previous foundry neuron design [35] also employs optoelectronics and a scalable MZI interconnect mesh, however, this design is not capable of the full range of neural behaviors described by the Izhikevich model. Using the GlobalFoundries (GF) 45SPCLO PDK, a new neuron was designed that can support a wider range of neural behaviors depending on applied voltage biasing. GF 45SPCLO is the successor of the GF 90WG PDK, and preserves the same CMOS-silicon photonic co-integration with a more advanced process node and additional metal routing layers. Fig. 1 shows the GF 45SPCLO neuron circuit design. The labeled red pins mark voltage biasing nodes that can be adjusted to achieve the desired neuron behavior. These nodes correspond to the control of an adjustable positive bias ($V_{bias}$), spiking threshold ($V_{th}$), refractory feedback rate ($V_{leak}$), and adaptation rate ($V_{leak2}$). The function of these node voltages is divided between membrane potential control and feedback potential control. Membrane potential controls $V_{bias}\;\&\;V_{th}$ adjust the spiking threshold and determine the current flow into membrane potential for each spike input. Feedback potential controls, $V_{leak}\;\&\;V_{leak2}$, determine the strength of negative feedback on the membrane potential and the length of refractory period. Balanced photodetectors receive excitatory and inhibitory light input. The diode at the circuit output incorporates the I-V characteristics of the laser diode chosen for the design.

To demonstrate this design, we first simulate the basic spiking behavior in response to excitatory and inhibitory inputs simulated in Cadence Spectre (shown in Figure 2). The nodes of each measurement are matched to the color of each line in Figure 1. We include inhibitory inputs on spike #11 and #12 and can confirm from Fig. 2 that inhibitory input suppressed the membrane potential and output, which matches our expectation.

Next, we demonstrate three spiking patterns: regular spiking (RS), fast spiking (FS), and chattering (CH) in analogy to [16]. These behaviors can be achieved flexibly by modifying the voltages at each biasing pin, which allows a greater tolerance for mismatch between design and tapeout. These spiking patterns are shown in Figure 3, Figure 4, and Figure 5 respectively. Input photocurrents are simulated as step functions from $0.0\,mA$ to $0.1\,mA$, node voltages corresponding to each behavior are set as follows:

1) Regular spiking: bias ($V_{bias}$) low, threshold ($V_{th}$) low, refractory feedback ($V_{leak}$) low, and frequency adaptation ($V_{leak2}$) low.

2) Fast spiking: bias ($V_{bias}$) low, threshold ($V_{th}$) high, refractory feedback ($V_{leak}$) low, and frequency adaptation ($V_{leak2}$) high.

3) Chattering: bias ($V_{bias}$) medium, threshold ($V_{th}$) medium, refractory feedback ($V_{leak}$) high, and frequency adaptation ($V_{leak2}$) high.

These simulations verify the ability of the neuron circuit to achieve various spiking patterns on the more advanced 45SPCLO process.

The building block of an MZI Mesh is a 4-port device that consists of two 50:50 beam splitters and two-phase shifters, $\theta$ and $\phi$ as shown in Fig. 6 (b). The phase shifter $\theta$, inside the interferometer, controls the power splitting ratio. Meanwhile, the phase shifter, $\phi$, outside of the interferometer, controls the relative phase difference between the two coherent input ports. As demonstrated in Fig 6 (a), the tunable power splitting functionality is tested by sweeping applied DC voltage on the phase shifter $\theta$. MZI Meshes can be arranged in several ways, with the most popular arrangements being the triangular[36] or rectangular[37] formations. Both of the formations can realize an arbitrary N$\times$N unitary matrix. There are a variety of applications where MZI Meshes are employed, such as mode-division multiplexing [38], free-space beamforming[39], quantum computing [40], and photonic neural networks [41]. Our work utilizes MZI Meshes as synaptic interconnections for bio-inspired neural networks and aims to integrate learning algorithms on the same chip. Although calibration procedures of the MZI Meshes are well-studied [42], training MZI Meshes as neural network (NN) interconnects remains challenging. Hughes et al. [43] proposed an in-situ training to realize the traditional backpropagation algorithm for MZI meshes, and recently Pai et al. [44] experimentally demonstrated the method. This in-situ training requires additional forward and backward light propagation with power monitoring for each phase shifter element at each step. There are various approaches to monitor power. For example, Pai et al. [44] utilized power tapping and grating couplers with an infrared camera to record the emitted power from MZI Meshes. Alternatively, Morichetti et al. [45] used a non-invasive power sensing device introduced for silicon waveguides. We exploited 1:99 power taps and Ge photodetectors (PDs) which are available as an instance in Process Design Kit (PDK) elements of the active silicon photonic multi-project-wafer (MPW) runs from the AIM Photonic foundry. Fig. 6 (a) shows photocurrent changes on the monitoring PDs with respect to applied voltage on phase-shifter $\theta$. Although we used thermo-optics as a simple and practical phase-shifting mechanism, it is possible to utilize micro-electro-mechanical systems (MEMS) for even lower power consumption [46] in future designs.

Fig. 7 (b) shows the fabricated and tested 6$\times$6 rectangular MZI Mesh with power taps after each 2$\times$2 MZI unit as shown in Fig. 6 (b). At each output waveguide of the 6$\times$6 mesh, a micro ring resonator (MRR) add-drop filter is placed with a PD on the drop ports, allowing for output monitoring by either optical or electrical means. When the MRR is at resonance, the output can be monitored and accessed through the electrical interface during the training. Alternatively, the MRR resonance wavelength can be tuned to let the optical signal propagate after the MZI Mesh. In this way, multiple MZI Mesh layers can be cascaded for DNN-like implementation. All the components are available in AIM Photonic’s PDK v4.0. The device is wirebonded on a fanout printed-circuit board. A USB interfaced multi-channel high current output digital to analog converter (DAC) unit drives the thermo-optic heaters and MRR add-drop filters. Similarly, the photocurrents are digitized by a USB interfaced multi-channel 250kSps analog-to-digital converter (ADC), as shown in Fig. 7 (a).

For the on-chip training demonstration, we targeted a linear classification problem with 4-dimensional input vectors and two output classes. We used the Iris flower dataset [47], consisting of 3 classes and 150 input samples. For simplicity in the proof-of-principle demonstration, we excluded one of the classes that is linearly separable from the other two classes. Therefore, a linear regression classifier can achieve a maximum of 94 true classifications over 100 samples. We use a single-mode cleaved fiber to couple a CW tunable laser source operating at $1553.7\,nm$ to the chip. After the edge coupler, the first three 2$\times$2 MZI stages act as tunable beam splitters and were used to generate coherent input vectors. First, the input generator phase-shifters are optimized adaptively to create desired 100 input samples. Next, optimum voltage values are recorded in a look-up table (LUT) to recall in the training and interference cycles.

One of the challenges of using MZI Meshes as a synaptic weight matrix is that controllable variables (phase shifters) do not explicitly map to individual weight matrix entries. In other words, adjusting a single phase shifter will affect multiple weight matrix entries. Clements et al. [37] devised a decomposition method for rectangular meshes. In machine learning, however, the optimum weight matrix is unknown at the beginning of training and the additional resources for continual adjustment and decomposition become intractable. Hughes et al. [43] demonstrated a method of differentiating the weight matrix w.r.t. each phase shifter. However, this method requires two optical propagation steps in addition to the initial inference step: one forward, and one backward. Therefore, an external controller is required to schedule each propagation, and light sources must be bidirectional. Moreover, during the additional optical propagation steps, power must be monitored for every phase shifter element. The number of phase shifters in the MZI mesh scales as $N(N-1)$ for N$\times$N weight matrices, meaning $\mathcal{O}$($N^{2}$) power monitoring is required. This presents remaining challenges for scalability in deep neural networks.

Here, we looked for more hardware-friendly solutions and, taking inspiration from biology, explored random backpropagation (RBP) and contrastive Hebbian learning (CHL) for MZI Meshes. In Section III-C1 we present an experimental demonstration of random backpropagation training for a linear classification task; Section III-C2 discusses the CHL algorithm and its relevance to human-like predictive error-driven learning.

The nanophotonic-electronic spiking neuron is composed of three main components: a photodetector, a nonlinear electrical circuit, and a laser. The photodetector receives information from the synaptic network and converts the optical signal to an electrical signal. The electrical circuit is the core of the neuron and processes the inputs to generate output spike responses. The laser output regenerates signal power after each layer to supply synaptic fanout to subsequent layers. Our team will exploit attojoule photonics with quantum impedance conversion [53] and closely integrate with low-capacitance ($<1\,fF$) electronics for monolithic integration on a silicon-on-insulator (SOI) platform. Using photonic communication between each SNN layer reduces capacitive charge associated with the interconnect wires [54] in comparable electronic circuits. Additionally, the photonic platform can allow neurons to communicate with other neurons at high speeds ($\sim$ 10 GHz) independently of communication distance.

To calculate the projected energy consumption, we can examine the composition of each component in the attojoule nanophotonic-electronic spiking neuron design. The dynamic energy cost of the nonlinear electronic circuit and laser can be calculated by examining the transistor on-state currents and associated operation voltages and frequency. Meanwhile the parasitic energy cost can be calculated from the total capacitance and the leakage current. According to our previous work[35], the electrical circuit current flow inside the maximum $10\,GHz$ spiking rate attojoule neuron is expected to be $31.27\,\mu A$ at $1.4\,V$ voltage supply when the neuron is in the ON state, while the leakage current is $10\,nA$ in the OFF state. The expected nanolaser energy consumption is  4.4 fJ per spike for a fanout of $\sim$ 80 [54]. The parasitic capacitance includes the load capacitance on the photodetector, membrane capacitor, and transistor gate capacitance. According to the IRDS2020 [55] and [56], we expect the load capacitance of the photodetector to be around 0.1fF, and the simulated membrane capacitor to be 0.5fF. By considering closely integrated nanoelectronics at 10 fJ/bit energy efficiency and a fanout of 10-100 following the concept outlined by [54], the minimum dynamic input energy to generate a spike output is projected to be 200aJ/spike.

For input, the proposed attojoule neuron design will utilize a low-Q nanophotonic crystal photodector with a Ge/Si cavity. The photonic crystal creates a resonant cavity that increases the confinement of light and reduces the size of the absorption medium [57][58]. This allows for an ultra-low capacitance ($\sim$0.1fF) nano-cavity PD that can generate sufficiently large voltage without amplification when combined with a high-impedance load [59]. In addition, minimizing the electrical wiring between PDs and the nonlinear electronic circuit also reduces power consumption [54]. Similarly for spiking output, a hybrid InAs/AlGaAs quantum-dot nanolaser with photonic crystal cavity can be employed.

Aside from neuron design, the scalability on interconnect is also a critical design challenge. MZI meshes show nearly lossless multiplication that is particularly suitable for large-scale low-power neuromorphic computing. However, the number of tunable elements, $N\cdot(N-1)$ in an $N\times N$ MZI mesh, grows polynomially with the number of neurons in the layer. As such, a control circuit must be designed that scales with a minimal additional computational complexity.

In the previous sections, we introduced and experimentally demonstrated bio-inspired on-chip training methods which improve the scalability of the SiPh MZI meshes for synaptic networks. We also simulated optoelectronic spiking neurons in GF 45SPCLO electronic-photonic hybrid platform and envisioned a scalable attojoule nanophotonic-electronic neuron design. However, one handicap of the proposed photonic neuromorphic system remains unaddressed, footprint efficiency. From our experience with commercial SiPh foundries, a 16$\times$16 MZI Meshes occupies a 12.5$mm^{2}$ chip area. Similarly, Lightmatter introduced their 64$\times$64 SiPh AI accelerator occupying a 150$mm^{2}$ chip area [60] which incorporates billions of transistors. We propose two solutions, Tensorized Photonic Neural Networks (TPNN) and 3D Electronic-Photonic Integrated Circuits (3D EPICs) to improve footprint efficiency and enable deep and wide photonic neuromorphic systems.

In relation to AI and machine learning, SNNs provide several advantages over modern computing paradigms for tasks which mimic the conditions in which they naturally evolved. Because SNNs process data over time in a continuous manner, they are well-suited to applications situated in real-time environments with single inference and learning instances presented at a time (such as event-based signal processing [74]). In addition, the spread of information over time allows multiple forms of memory at different time-scales similar to the human distinction between working [75], short-term [76], and long-term memories. Neuromorphic sensing and robotics are a common direction of applications of SNNs; for example, an adaptive robotic arm controller can provide reliable motor control as actuators wear down [77]. More speculatively, future devices might exploit these properties in the context of live audio and natural language processing for voice assistants, live-captioning services, or audio separation; similarly SNNs can be used for live video and lidar processing in autonomous vehicles or surveillance systems. SNNs are not ideal for batched computation—in which multiple training samples are computed in parallel and averaged for parallelism in training—however, data centers may still make use of the increased computational parallelism in tasks like the nearest-neighbor search which can be performs in constant time, $O(1)$, on neuromorphic chips like Loihi [78].

A major challenge of many modern DNN and reinforcement learning (RL) agents is the development of abstract, transformation-invariant representations of objects relevant to the task. In classification tasks, a neural network must transform its input space into a representation which most clearly separates each labeled class. Similarly, RL agents must be able to process their input space into a representation that best accentuates the value of potential actions. Predictive error-driven learning, modeled after the work of O’Reilly [52], has the potential to autonomously build deep hierarchies of abstraction for a given input space. For example, a learning agent could implicitly learn physical properties of the world such as gravity, buoyancy, and contact forces simply by observing its environment. In combination with complimentary learning systems for memory [79] and RL models based on the basal ganglia [80], a neuromorphic learning agent may be capable of replicating simple navigation and foraging behaviors which require the flexible application of knowledge and memory. Such a model could provide key insights for the development of self-motivated learning agents that exploit hierarchical representations to solve reinforced tasks. Developing dedicated spiking neuromorphic hardware and taking advantage of the energy-efficient and scalable photonic devices will allow the development of larger models and new computational paradigms. These developments can be applied in dynamic, noisy environments that are not well-handled by today’s machine learning efforts.