Effects of noise on leaky integrate-and-fire neuron models for neuromorphic computing applications

Let us recall the SNN algorithm, presented schematically in Fig. 1 (see, e.g., [10]). At the first step, pre-synaptic neuronal drivers provide the input voltage spikes. Then, we convert the input driver for spikes to a gently varying current signal proportional to the synaptic weights $w_{1}$ and $w_{2}$. Next, the synaptic current response is summed into the input of LIF neuron $N_{3}$. Then, the LIF neuron integrates the input current across a capacitor, which raises its potential. After that, $N_{3}$ resets immediately (i.e. loses stored charge) once the potential reaches/exceeds a threshold. Finally, every time $N_{3}$ reaches the threshold, a driver neuron D3 produces a spike.

In general, the biological neuronal network is related to the SNN algorithm. Moreover, the main role of SNNs is to understand and mimic human brain functionalities since SNNs enable to approximate efficient learning and recognition tasks in neuro-biology. Hence, to have a better implementation of SNNs in hardware, it would be necessary to describe an efficient analog of the biological neuron. Therefore, in what follows, we are interested in the SNN algorithm starting from the third step, where the synaptic current response is summed into the input of LIF neuron, to the last step of the SNN algorithm. In particular, at the third step of SNN algorithm, it is assumed that the summation of synaptic current responses can be a constant, in a deterministic form or can be even represented by a random type of current. To get closer to the real scenarios of neuronal models, we should also account for the existence of random fluctuations in the systems. Specifically, the random inputs arise primarily through sensory fluctuations, brainstem discharges and thermal energy (random fluctuations at a microscopic level, such as Brownian motions of ions). The stochasticity can arise even from the devices which are used for medical treatments, e.g. devices for injection currents into the neuronal systems. For simplicity, we consider a LIF synaptic conductance model with additive and multiplicative noise input currents in presence of a random refractory period.

In biological systems such as brain networks, instead of physically joined neurons, a spike in the presynaptic cell causes a chemical, or a neurotransmitter, to be released into a small space between the neurons called the synaptic cleft [14]. Therefore, in what follows, we will focus on investigating chemical synaptic transmission and study how excitation and inhibition affect the patterns in the neurons’ spiking output. In this section, we consider a model of synaptic conductance dynamics. In particular, neurons receive a myriad of excitatory and inhibitory synaptic inputs at dendrites. To better understand the mechanisms of synaptic conductance dynamics, we investigate the dynamics of the random excitatiory (E) and inhibitory inputs to a neuron [21].

In general, synaptic inputs are the combination of excitatory neurotransmitters. Such neurotransmitters depolarize the cell and drive it towards the spike threshold, while inhibitory neurotransmitters hyperpolarize it and drive it away from the spike threshold. These chemical factors cause specific ion channels on the postsynaptic neuron to open. Then, the results make a change in the neuron’s conductance. Therefore, the current will flow in or out of the cell [14].

For simplicity, we define transmitter-activated ion channels as an explicitly time-dependent conductivity $(g_{\text{syn}}(t))$. Such conductance transients can be generated by the following equation (see, e.g., [9, 14]):

where $\bar{g}_{\text{syn}}$ (synaptic weight) is the maximum conductance elicited by each incoming spike, while $\tau_{\text{syn}}$ is the synaptic time constant and $\delta(\cdot)$ is the Dirac delta function. Note that the summation runs over all spikes received by the neuron at time $t_{k}$. Using Ohm’s law, we have the following formula for converting conductance changes to the current:

where $E_{\text{syn}}$ represents the direction of current flow and the excitatory or inhibitory nature of the synapse.

In general, the total synaptic input current $I_{\text{syn}}$ is the sum of both excitatory and inhibitory inputs. We assume that the total excitatory and inhibitory conductances received at time $t$ are $g_{E}(t)$ and $g_{I}(t)$, and their corresponding reversal potentials are $E_{E}$ and $E_{I}$, respectively. We define the total synaptic current by the following equation:

Therefore, the corresponding membrane potential dynamics of the LIF neuron under synaptic current (see, e.g., [21]) can be described as follows:

where $V$ is the membrane potential, $I_{\text{inj}}$ is the external input current, while $\tau_{m}$ is the membrane time constant. We consider the membrane potential model where a spike takes place whenever $V(t)$ crosses $V_{\text{th}}$. Here, $V_{\text{th}}$ denotes the membrane potential threshold to fire an action potential. In that case, a spike is recorded and $V(t)$ resets to $V_{\text{reset}}$ value. This is summarized in the reset condition $V(t)=V_{\text{reset}}$ if $V(t)\geq V_{\text{th}}$. We define the following LIF model with and a reset condition:

In this model, we consider a random synaptic input by introducing the following random input current (additive noise) $I_{\text{inj}}=I_{0}+\sigma_{1}\eta(t)$, where $\eta$ is the zero-mean Gaussian white noise with unit variance. For the multiplicative noise case, the applied current is set to $I_{\text{inj}}=V(t)(I_{0}+\sigma_{2}\eta(t))$. Here, $\sigma_{1},\sigma_{2}$ denote the standard deviations of these random components to the inputs. When considering such random input currents, the equation (4) can be considered as the following Langevin stochastic equation (see, e.g., [25]):

In our model, we use the simplest input spikes with Poisson process which provide a suitable approximation to stochastic neuronal firings [29]. This input spikes will be added in the quantity $\sum_{k}\delta(t-t_{k})$ in the equation (1). In particular, the input spikes are given when every input spike arrives independently of other spikes. For designing a spike generator of spike train, let us call the probability of firing a spike within a short interval (see, e.g. [9]) $P(1\text{ spike during }\Delta t)=r_{j}\Delta t$, where $j=e,i$ with $r_{e},r_{i}$ representing the instantaneous excitatory and inhibitory firing rates, respectively. This expression is designed to generate a Poisson spike train by first subdividing time into a group of short intervals through small time steps $\Delta t$. At each time step, we define a random variable $x_{\text{rand}}$ with uniform distribution over the range between 0 and 1. Then, we compare this with the probability of firing a spike, which is described as follows:

In this work, we also investigate the effects of random refractory periods [22]. We define the random refractory periods $t_{\text{ref}}$ as $t_{\text{ref}}=\mu_{\text{ref}}+\sigma_{\text{ref}}\tilde{\eta}(t)$, where $\tilde{\eta}(t)\sim\mathcal{N}(0,1)$.

In general, the irregularity of spike trains can provide information about stimulating activities in a neuron. A LIF synaptic conductance neuron with multiple inputs and coefficient of variation (CV) of the inter-spike-interval (ISI) can bring an output decoded neuron. In this work, we show that the increase $\sigma_{\text{ref}}$ can lead to an increase in the irregularity of the spike trains (see also [13]).

We define the spike regularity via coefficient of variation of the inter-spike-interval (see, e.g., [8, 13]) as follows:

where $\sigma_{\text{ISI}}$ is the standard deviation and $\mu_{\text{ISI}}$ is the mean of the ISI of an individual neuron.

In the next section, we consider the output firing rate as a function of Gaussian white noise mean or direct current value, known as the input-output transfer function of the neuron.

In Fig. 3, we have plotted the Gaussian white noise current profile, the time evolution of the membrane potential $V(t)$ with Gaussian white noise input current ($I_{\text{inj}}=200+\eta(t)$ (pA)) and direct input current ($I_{\text{inj}}=I_{\text{dc}}=200$ (pA)). In this case, we fix the value of $t_{\text{ref}}=8+2\tilde{\eta}(t)$ (ms). We observe that the time evolution of the membrane potential looks quite similar in the two cases. Note that a burst occurred when a neuron spiked more than once within 25 (ms) (see, e.g., [26]). In this case, when considering the presence of random input current and random refractory period in the system, we observe there exist bursts in the case presented in the second row of Fig. 3.

We look also at the input-output transfer function of the neuron, the output firing as a function of average injected current in Fig. 4. In particular, we see that the spike count values are slightly fluctuating when we add the random refractory period into the system (in the right panel of Fig. 4). Moreover, in the averaged injected current intervals $[130;290]$ (pA) and $[320;360]$ (pA), the spike count value is the same in both cases: the Gaussian white noise and direct currents in the right panel of Fig. 4. We have seen this phenomenon for $I_{\text{inj}}=200$ (pA) also in the Fig. 3.

In Fig.5, we have plotted the spike count distribution as a function of ISI. We observe that the coefficient $CV_{\text{isi}}$ increases when we increase the value of $\sigma_{\text{ref}}$. The spikes are distributed almost entirely in the ISI interval from 6 to 9 (ms) in the first row, while the spikes are distributed mostly in from 0 to 21 (ms) in the second row of Fig.5. The ISI distribution presented in the first row of Fig.5 reflects bursting moods. To understand better such phenomenon let us look at the following spike irregularity profile in Fig. 6.

In Fig.6, we look at the corresponding spike irregularity profile of the spike count in Figs. 5-6. In this plot, we fix the external current $I_{\text{inj}}=200+\eta(t)$ (pA) together with considering different values of $\sigma_{\text{ref}}$. We observe that when we increase the value of $\sigma_{\text{ref}}$ the coefficient $CV_{\text{isi}}$ increases. In general, when we increase the mean of the Gaussian white noise, at some point, the effective input means are above the spike threshold and then the neuron operates in the so-called mean-driven regime. Hence, as the input is sufficiently high, the neuron is charged up to the spike threshold and then it is reset. This essentially gives an almost regular spiking. However, in our case, by considering various values of the random refractory period, we see that $CV_{\text{isi}}$ increases when we increase the values of $\sigma_{\text{ref}}$. This is visible in Fig. 6, $CV_{\text{isi}}$ increases from 0.1 to 0.6 when $\sigma_{\text{ref}}$ increases from 1 to 6. Note that an increased ISI regularity could result in bursting [23]. Moreover, the spike trains are substantially more regular with a range $CV_{\text{ISI}}\in(0;0.5)$, and more irregular when $CV_{\text{ISI}}>0.5$ [27]. Therefore, in some cases, the presence of random input current with oscilations could lead to the burst discharge.

In Fig. 7, we have plotted the Gaussian white noise current profile, the time evolution of the membrane potential $V(t)$ with multiplicative noise input current ($I_{\text{inj}}=V(t)(200+\eta(t))$ (pA)) and direct input current ($I_{\text{inj}}=I_{\text{dc}}=200V(t)$ (pA)). In this case, we fix the value of $t_{\text{ref}}=8+\eta(t)$ (ms). There are bursting moods in the membrane potential in both two cases. This is due to the presence of $V(t)$ in the input current together with the random refractory period in the system.

However, when we increase the leak conductance from $g_{L}=20$ (nS) to $g_{L}=200$ (nS), the burst discharges are dramatically reduced in the case with multiplicative noise in Fig. 8. In particular, we observe fluctuations in the membrane potential in the second row of Fig. 8. There is an increase in the time interval between two nearest neighbor spikes in both cases. In order to understand better such phenomena, let us look at the following plots. From now on, we will use the parameter $g_{L}=200$ (nS) for cases in Figs. 9-11.

In Fig. 9, we look at the input-output transfer function of the neuron, output firing rate as a function of input means. We observe that the input-output transfer function looks quite similar in both cases: direct and random refractory periods. There are slight fluctuations in the spike count profile in the case with random refractory period. It is clear that the presence of multiplicative noise strongly affects the spiking activity in our system compared to the case with additive noise. The spiking activity of the neuron dramatically reduces in presence of the multiplicative noise in the system.

In Fig. 10, we have plotted the spike distribution as a function of ISI. In the first row of Fig. 10, we see that the spikes are distributed almost entirely in the ISI interval [9;18] (ms). Moreover, when we increase the value of $\sigma_{\text{ref}}$ from 0.5 to 4 the spike irregularity values increase. In addition, they are distributed almost entirely in the ISI interval [1;25] (ms) in the second row of Fig. 10. To understand better such phenomenon we look at the spike irregularity profile of our system in presence of multiplicative noise and random refractory period in Fig. 11. In particular, we observe that the spike irregularity $CV_{\text{ISI}}$ increases when we increase the values of $\sigma_{\text{ref}}$ similar to the case of additive noise. Furthermore, we see that the larger injected currents are, the higher are the values of $CV_{\text{ISI}}$.

Additionally, we notice that the presence of the random refractory period increases the spiking activity of the neuron. The presence of additive and multiplicative noise causes burst discharges in the system. However, when we increase the value of leak conductance, the burst discharges are strongly reduced in the case with multiplicative noise. Under suitable values of average injected current as well as the values of random input current and random refractory period, the irregularity of spike trains increases. The presence of additive noise could lead to the occurrence of bursts, while the presence of multiplicative noise with random refractory period could reduce the burst discharges in some cases. This effect may lead to an improvement in the carrying of information about stimulating activities in the neuron [1]. Moreover, the study of random factors in the LIF conductance model would potentially contribute to further progress in addressing the challenge of how the active membrane currents generating bursts of action potentials affect neural coding and computation [19]. Finally, we remark that noise may come from different sources, e.g., devices, environment, chemical reactions. Moreover, as such, noise is not always a problem for neurons, it can also bring benefits to nervous systems [11, 1].