Neuromorphic Integrated Sensing and Communications

IR is widely used for low-power communications, e.g., in the IEEE 802.15.4z standard [8], and is envisaged to be part of the “gearbox” of the physical layer of 6G systems [9]. Among the key advantages of IR is the possibility to implement receivers that can operate using efficient analog or neuromorphic hardware [10, 11, 12]. The integration of IR and neuromorphic computing was investigated in [5, 6], which proposed an end-to-end neuromorphic architecture for remote inference that replaces traditional digital blocks with SNNs as encoder and decoder. Hardware implementations includes [10], which introduced an IR-based communication protocol to convey digital packets between SNN chips (see also [13]); and reference [11], which proposed an all-digital spike-based IR wireless transmission scheme for miniaturized biomedical applications.

One of the key research directions for ISAC systems is optimal waveform design [14]. For instance, reference [15] studied ISAC based on delay alignment modulation techniques for Terahertz massive MIMO.

To our best of knowledge, this is the first work to propose the implementation of ISAC via neuromorphic computing. The main contributions of this letter are summarized as follows.

• •

  We introduce the novel N-ISAC system illustrated in Fig. 2, which exploits neuromorphic computing and IR to achieve simultaneous data transmission and target detection. The proposed SNN-based architecture demodulates digital data and detects the presence of a radar target in an online fashion;

• •

  We propose a supervised learning method for the design of the N-ISAC receiver in Fig. 2;

• •

  Numerical results are provided that demonstrate the advantages of the proposed N-ISAC system over conventional separate sensing and communication (SSAC) solutions.

The remainder of the paper is organized as follows. Section II presents the system model. The neuromorphic receiver processing is detailed in Section III, while Section IV presents the experimental setting and results. Finally, Section V concludes the paper.

The IR transmitter modulates and transmits digital data $x_{l}\in\{0,1\}$, for each discrete-time instant $l=1,\ldots,L$, using pulse position modulation (PPM) through a single antenna. Each bit $x_{l}$ is encoded within the $l$th time slot of duration $T$ seconds. Specifically, given an information bit sequence $\{x_{1},\ldots,x_{L}\}$ of $L$ bits, the PPM-modulated signal is given as [16]

$\displaystyle s(t)=\sum_{l=1}^{L}\phi\big{(}t-(l-1)T-x_{l}L_{b}T_{c}\big{)},$

(1)

where $\phi(t)$ is the pulse waveform of bandwidth $1/T_{c}$; and $T_{c}$ is the chip time. The number of chips within each slot is $T/T_{c}=2L_{b}$, where integer $L_{b}\geq 1$ is referred to as the bandwidth expansion factor. Accordingly, as illustrated in Fig. 3, in each slot $l$, bit $x_{l}$ is encoded by a pulse on the first chip when $x_{l}=0$, and by a pulse in the middle of the slot, i.e., at the $L_{b}+1$th chip, when $x_{l}=1$. The energy of the pulse waveform $\phi(t)$ is constrained to be smaller than a given value $E_{b}$, i.e., $\int_{-\infty}^{+\infty}\phi^{2}(t)\leq E_{b}$.

The modulated signal $s(t)$ is transmitted over a multi-path fading channel to the receiver. The multi-path channel depends on the possible presence of a radar target, as well as on scatterers located between the transmitter and receiver that act as clutter for radar processing. We use the binary variable $v$ to indicate the presence or absence of the radar target, identified respectively by setting $v=1$ and $v=0$. Specifically, following a standard radar model (see, e.g., [17]), we assume that a target may or may not be present at a radar cell corresponding to a known propagation delay $\tau_{0}$. Accordingly, the continuous-time channel response can be expressed as

$\displaystyle h(t)=v\beta_{0}g(t-\tau_{0})+\sum_{c=1}^{N_{c}}\beta_{c}g(t-\tau_{c}),$

(2)

where $g(t)=\phi(-t)$ is the receiver filter response, which is matched to the transmitted waveform $\phi(t)$; the amplitude $\beta_{0}$ of the target follows a complex Gaussian distribution with power $\sigma_{0}^{2}$, i.e., $\beta_{0}\sim\mathcal{CN}(0,\sigma_{0}^{2})$; and the amplitudes $\{\beta_{c}\}_{c=1}^{N_{c}}$ of the $N_{c}$ clutter components are independent with uniform phases and Weibull absolute values having shape parameter $\kappa\in[0.25,2]$ and scale parameter $\lambda\in(0,\infty)$ [17]. With $\kappa=2$, the amplitudes $\{\beta_{c}\}_{c=1}^{N_{C}}$ are complex Gaussian random variables [17]. The amplitudes $\beta_{0}$ and $\{\beta_{c}\}_{c=1}^{N_{c}}$ and delays $\{\tau_{c}\}_{c=1}^{N_{c}}$ are assumed to be unknown.

We note that the model (2) could be generalized to allow for a time-varying presence/absence pattern for the target, in the sense that the variable $v$ could change over the time slot index $l$. We will not consider this situation in this paper, although the extension is straightforward.

The signal obtained by the single-antenna receiver is given by

$\displaystyle y(t)=s(t)*h(t)+z(t),$

(3)

where the channel response $h(t)$ is given in (2); $z(t)$ is additive white Gaussian noise with power spectral density $N_{0}$; and “$*$” denotes the convolution operation. The receiver samples the received signal $y(t)$ at the chip rate $1/T_{c}$, yielding the discrete-time signal $y_{i}=y(iT_{c})$, for $i=1,2,\ldots,2L_{b}L$. For each slot time $l$, we collect the $2L_{b}$ samples $\mbox{\boldmath{$y$}}_{l}=\{y_{i}\}_{i\in\mathcal{I}_{l}}$, with $i\in\mathcal{I}_{l}=\{2(l-1)L_{b}+1,\ldots,2lL_{b}\}$. Assuming a maximum delay spread $T_{h}$, upon sampling, the effective discrete-time channel has $L_{h}=T_{h}/T_{c}$ taps. Accordingly, the channel between the transmitter and the receiver is described by the $L_{h}\times 1$ vector $\mbox{\boldmath{$h$}}=[h(0),h(T_{c}),\ldots,h((L_{h}-1)T_{c})]^{T}$. Note that, if $L_{h}<L_{b}$, there is no interference between pulses transmitted in successive slots; while, otherwise, transmission of the pulse encoding the $l$th bit $x_{l}$ may interfere with the pulse encoding the following bits $\{x_{l^{\prime}}\}_{l^{\prime}>l}$.

Using (3), the received discrete-time signal $y_{i}$ for the $i$th chip can be expressed as

$\displaystyle y_{i}=\mbox{\boldmath{$h$}}^{T}\mbox{\boldmath{$s$}}_{i}+z_{i},$

(4)

where we have defined the $L_{h}\times 1$ vector $\mbox{\boldmath{$s$}}_{i}=[s_{i},s_{i-1},\ldots,\\ s_{i-L_{h}+1}]^{T}$, with $s_{i}=s(iT_{c})$ and $s_{i}=0$ if $i\leq 0$; and $z_{i}=z_{n}(iT_{c})\sim\mathcal{CN}(0,N_{0}B)$ are the noise samples, where $B=1/T_{c}$ is the pulse bandwidth. By the assumed PPM scheme, for each slot $l$, we have all zero samples $\{s_{i}\}_{i\in\mathcal{I}_{l}}$ except for one sample, namely $s_{2(l-1)L_{b}+1}=1$ if $x_{l}=0$ and $s_{2(l-1/2)L_{b}+1}=1$ if $x_{l}=1$.

The receiver feeds the received samples $\{y_{i}\}_{i\in\mathcal{I}}$ to the SNN successively for each slot $l$. To this end, we collect the received samples $\{y_{i}\}_{i\in\mathcal{I}}$ corresponding to the time slot $l$ into a $2L_{b}\times 1$ vector $\mbox{\boldmath{$y$}}_{l}$. Since $\mbox{\boldmath{$y$}}_{l}$ is generally complex, we define the $4L_{b}\times 1$ real-value vector

$\displaystyle\bar{\mbox{\boldmath{$y$}}}_{l}=[\mathfrak{R}(\mbox{\boldmath{$y$}}_{l})^{T},~{}\mathfrak{S}(\mbox{\boldmath{$y$}}_{l})^{T}]^{T},$

(5)

with $\mathfrak{R}(\cdot)$ and $\mathfrak{S}(\cdot)$ being the element-wise real and imaginary parts of the input vector, respectively. The received signal $\bar{\mbox{\boldmath{$y$}}}_{l}$ is input to the SNN at each time step $l$.

As described in the next section, an SNN is a recurrent discrete-time model that operates along the slot index $l$. It takes as input the $2L_{b}$ real samples $\bar{\mbox{\boldmath{$y$}}}_{l}$, with one output neuron producing the decoding output $\hat{x}_{l}\in\{0,1\}$, which provides an estimate of the data symbol $x_{l}$, and the other output neuron producing the sensing output $\hat{v}_{l}$, which estimates the binary variable $v_{l}\in\{0,1\}$ representing the presence or absence of the target. Note that the detection decision $\hat{v}_{l}$ about the target varies over time as more information is acquired by the receiver.

An SNN is specified as a network connecting a set of spiking neurons via an arbitrary directed graph, in which a directed edge represents a synapse. To model the spiking neurons, we adopt the standard discrete-time spike response model (SRM), in which each spiking neuron outputs a binary signal $b_{l}\in\{0,1\}$, with “1” representing the emission of a spike and “0” an idle neuron at time step $l=1,\ldots,L$. Each neuron $k$ maintains an internal analog state variable $o_{k,l}$, known as the membrane potential, over time step $l$.

The membrane potential $o_{k,l}$ evolves over time $l=1,2,\ldots$ as a function of the spikes that neuron $k$ receives from the neurons that have synapses ending at neuron $k$. The neuron spikes when the membrane potential crosses a threshold, after which the membrane potential is reduced below the threshold. Mathematically, the membrane potential $o_{k,l}$ is defined by the sum of filtered contributions from incoming spikes and from the neuron $k$’s own past outputs. Accordingly, the evolution of the membrane potential is modelled as

$\displaystyle o_{k,l}=\sum_{j\in\mathcal{P}_{k}}w_{k,j}\cdot(\alpha_{l}*b_{k,l})+\beta_{l}*b_{k,l},$

(6)

where $\mathcal{P}_{k}$ is the set of neurons with synapses ending at neuron $k$; $w_{k,j}$ is the synaptic weight between neuron $j\in\mathcal{P}_{k}$ and neuron $k$; $\alpha_{l}$ represents the synaptic response to a spike from a neuron $j\in\mathcal{P}_{k}$; $\beta_{l}$ describes the synaptic response to the spike emitted by neuron $k$ itself; and “$*$” is the convolution operator. Typical choices for synaptic spike responses include the first-order feedback filter $\beta_{l}=\exp(-l/\tau_{\rm ref})$, and the second-order synaptic filer $\alpha_{l}=\exp(-l/{\tau_{\rm mem}})-\exp(-l/{\tau_{\rm syn}})$, for $l=1,2,\ldots$, with finite positive constants $\tau_{\rm ref}$, $\tau_{\rm mem}$ and $\tau_{\rm syn}$. We refer to [18, 19] for further details.

Neuron $k$ outputs a spike at time step $l$ if its membrane potential $o_{k,l}$ passes some fixed threshold $\vartheta$. Accordingly, the output of neuron $k$ can be expressed as

$\displaystyle b_{k,l}=\Theta(o_{k,l}-\vartheta),$

(7)

where $\Theta(\cdot)$ is the Heaviside step function ($\Theta(x)=1$ if $x>0$ and $\Theta(x)=0$ otherwise). We denote as $\theta$ the vector of all parameters $\{w_{k,j}\}$ for the SNN of the receiver.

The SNN at the receiver has two read-out neurons producing decisions $\hat{x}_{l}$ and $\hat{v}_{l}$. Denoting as $k^{c}$ the index of the communication readout neuron, and as $k^{s}$ the index of the radar readout neuron, by (7) we have the estimates $\hat{x}_{l}=b_{k^{c},l}$ and $\hat{v}_{l}=b_{k^{s},l}$.

To optimize the model parameters of the SNN, we adopt a supervised learning approach, and we assume the availability of a dataset $\mathcal{D}$ consisting of examples of the form $(\bar{\mbox{\boldmath{$y$}}},\mbox{\boldmath{$x$}},v)$. Each example $(\bar{\mbox{\boldmath{$y$}}},\mbox{\boldmath{$x$}},v)$ contains the sequence of $L$ transmitted bits $\mbox{\boldmath{$x$}}=\{x_{l}\}_{l=1}^{L}$ with $x_{l}\in\{0,1\}$; the corresponding $L$ received samples $\bar{\mbox{\boldmath{$y$}}}=\{\bar{\mbox{\boldmath{$y$}}}_{l}\}_{l=1}^{L}$ obtained via (5); and the binary value $v$ indicating the presence or absence of the target, which is assumed to be the same across $L$ samples. For each example $(\bar{\mbox{\boldmath{$y$}}},\mbox{\boldmath{$x$}},v)$, at each $l$th step, the received signal $\bar{\mbox{\boldmath{$y$}}}_{l}$ serves as input to the SNN, while $x_{l}$ and $v$ are ground-truth labels for data decoding and target detection, respectively.

To define the training criterion, we evaluate the performance of communication and sensing via two separate cross-entropy losses. To this end, defining the sigmoid function $\sigma(x)=(1+e^{-x})^{-1}$, we write the probability $p^{c}_{l}(\mbox{\boldmath{$\theta$}})=\sigma(o_{k^{c},l})$ of the communication readout neuron to produce the estimate $\hat{x}_{l}=1$; and the probability $p^{s}_{l}(\mbox{\boldmath{$\theta$}})=\sigma(o_{k^{s},l})$ of the radar readout neuron to produce the estimate $\hat{v}_{l}=1$. They are both functions of the model parameters $\theta$. The training loss related to data decoding is then measured by the cross-entropy loss

$\displaystyle\ell^{c}_{l}(\mbox{\boldmath{$\theta$}})=-x_{l}\log(p_{l}^{c}(\mbox{\boldmath{$\theta$}}))-(1-x_{l})\log(1-p_{l}^{c}(\mbox{\boldmath{$\theta$}})),$

(8)

while the training loss for sensing is similarly defined as

$\displaystyle\ell^{s}_{l}(\mbox{\boldmath{$\theta$}})=-v\log(p_{l}^{s}(\mbox{\boldmath{$\theta$}}))-(1-v)\log(1-p_{l}^{s}(\mbox{\boldmath{$\theta$}})).$

(9)

The overall training losses for each data point are defined as the sums over time $L^{c}(\mbox{\boldmath{$\theta$}})=\sum_{l=1}^{L}\ell_{l}^{c}(\mbox{\boldmath{$\theta$}})$ and $L^{s}(\mbox{\boldmath{$\theta$}})=\sum_{l=1}^{L}\ell_{l}^{s}(\mbox{\boldmath{$\theta$}})$. For the considered SNN architecture, we adopt the weighted sum

$\displaystyle L^{ISAC}(\mbox{\boldmath{$\theta$}})=\beta L^{c}(\mbox{\boldmath{$\theta$}})+(1-\beta)L^{s}(\mbox{\boldmath{$\theta$}})$

(10)

of the communication and sensing losses, where $\beta\in[0,1]$ is a weight factor determining the relative priority between the two losses. The model parameters $\theta$ are updated based on stochastic gradient descent (SGD). To this end, we address the nondifferentiable threshold activation (7) via surrogate gradient [20] by replacing the Heaviside step function in (7) with a differentiable surrogate function, namely the sigmoid $\sigma(x)$, during training.

The system operates over $L=80$ slots. The information bits $\{x_{l}\}_{l=1}^{L}$ and target $v$ are i.i.d. generated as $1/2$-Bernoulli variables, i.e., $x_{l}\sim\text{Bern}(0.5)$, and $v\sim\text{Bern}(0.5)$. The target delay is set to $\tau_{0}=0$, and the amplitudes $\{\beta_{c}\}_{c=0}^{N_{c}}$ are all i.i.d. $\beta_{c}\sim\mathcal{CN}(0,1)$ variables, while the delays $\{\tau_{c}\}_{c=1}^{N_{c}}$ are uniformly distributed between 0 and $4T_{c}$. We set the number of clutter paths in the channel (2) as $N_{c}=5$. We generate $60,000$ training examples and $10,000$ test examples as described in Section II-B. The SNN has a fully connected architecture with a single hidden layer, with $4L_{b}$ input neurons and $6$ or $10$ hidden neurons. The average signal-to-noise ratio (SNR) is defined as $\mathrm{SNR}=\mathbb{E}[\|\mbox{\boldmath{$h$}}\|^{2}]E_{b}/(N_{0}B)$, where $\mathbb{E}[\|\mbox{\boldmath{$h$}}\|^{2}]$ is the average squared norm of the channel. We set the SNR to $10$dB.

For comparison, we consider a conventional SSAC scheme. SSAC divides the $L$ slots into $\lceil\alpha L\rceil$ slots used to transmit information, and $L-\lceil\alpha L\rceil$ slots used for radar sensing, where $\alpha\in[0,1]$. The transmitted signal for radar sensing is given by the PPM waveform (1) with bits fixed to $\{x_{l}=1\}_{l=\lceil\alpha L\rceil+1}^{L}$. For SSAC, two SNNs are implemented at the receiver, one performing data decoding for the first $\lceil\alpha L\rceil$ slots, and the other responsible for radar sensing in the rest of the time slots. The architecture of both SNNs is as described above, with each SNN having $6$ hidden neurons.

We adopt the following performance metrics for data transmission and radar sensing:

• •

  Normalized test throughput, i.e., the ratio $\mathbb{E}[L_{\rm succ}]/L$ of the average number $L_{\rm succ}<L$ of correctly decoded bits over the total number of time slots $L$;

• •

  Radar test detection error, i.e., the probability that the sensing decision $\hat{v}$ is not correctly taken upon processing all time slots. The final decision $\hat{v}$ for each example is made based on the majority rule, predicting the presence of a target $(\hat{v}=1)$ if the SNN produces the decision $\hat{v}_{l}=1$ for a majority of the time slots $l=1,\ldots,L$.

We demonstrate the normalized test throughput versus the radar test detection error for ISAC and SSAC when there is no bandwidth expansion, e.g., for $L_{b}=1$, in Fig. 4. For the ISAC scheme, we vary $\beta$ in the loss (10), while we change the fraction $\alpha$ for SSAC. As $\beta$ increases, more priority is given by ISAC to communication over radar detection; and, similarly, as $\alpha$ increases, SSAC assigns more slots to communications. The performance of ISAC with an SNN having $10$ hidden neurons is essentially independent of $\beta$ for any $0.25<\beta<0.75$. A first observation is that, for SSAC, there is a trade-off between communication and sensing performance levels caused by the slot allocation. A similar trade-off is also observed for ISAC when using an SNN with $6$ hidden neurons. This is due to the limited capacity of the shared common hidden layer of the SNN. In contrast, when $10$ hidden neurons are available at the SNN, ISAC is seen to optimize both data decoding and target sensing performance, obtaining significant gains over SSAC.

In Fig. 5, we demonstrate the normalized test throughput and radar test detection error versus the bandwidth expansion factor $L_{b}$ for ISAC and SSAC by setting $\alpha=\beta=0.5$. Both ISAC and SSAC are observed to benefit from the bandwidth expansion due to the reduced inter-slot interference. However, the throughput achievable by SSAC is bounded by $\alpha=0.5$, while ISAC can obtain larger throughputs; and a similar behavior is observed also in terms of the probability of detection error. Furthermore, when the bandwidth expansion $L_{b}$ is sufficiently large, e.g., when $L_{b}=6$, ISAC with $6$ hidden neurons obtains a similar performance as that with $10$ neurons, which indicates a larger bandwidth simplifies receiver design in terms of SNN’s capacity.

Finally, Fig. 6 illustrates how the SNN receiver can leverage the temporal sparsity of the IR signals to enhance energy efficiency. In this regard, we recall that energy consumption in an SNN is essentially proportional to the number of spikes produced by the SNN, given extremely low idle energy of neuromorphic chips [7]. The top panel shows the transmitted IR signal consisting of two frames of transmitted signals with $L_{b}=1$, separated by an idle frame of duration of $20$ slots. We observe that in the idle frame, the spike count is significantly reduced, showing that the neuromorphic receiver can adjust its energy consumption to the activity level of the transmitter.