Toward Ambient Intelligence: Federated Edge Learning with Task-Oriented Sensing, Computation, and Communication Integration

As shown in Fig. 2, we consider an ISAC-assisted FEEL system comprising $K$ ISAC devices, denoted by a set $\mathcal{K}=\{1,2,...,K\}$, and a single edge server. Each ISAC device is equipped with a single-antenna ISAC transceiver that can switch between the sensing mode and communication mode as needed in a time-devision manner using a shared radio-frequency front-end circuit¹¹1A practical implementation of such an ISAC device via software-defined radio platform has been demonstrated in [24]. [24]. Specifically, in the sensing mode, dedicated radar waveform known as frequency-modulated continuous-wave (FMCW) consisting of multiple up-chirps is transmitted [17]. Then, by processing the received radar echo signals, sensing data that contain the motion information of the human target can be attained at ISAC devices (see Sec. III-B for more details of the sensing processing). On the other hand, in the communication mode, constant-frequency carrier modulated by communication data using digital modulation scheme (e.g., QAM) is transmitted. This mode is used for the necessary information exchange between the ISAC device and edge server in the course of FEEL as elaborated in Sec. II-B. For better illustration, the time-frequency diagram for the considered time-switching ISAC waveforms is depicted in Fig. 3. Besides, each ISAC device is equipped with certain computation power and can perform local computations for training and inference of ML models, e.g., convolutional neural network (CNN). The goal of this ISAC-assisted FEEL system is to train²²2After the model training stage is completed, each device will deploy the model locally for inference. In this work, we only focus on the optimization in the training stage. an ML model tailored for specific sensing applications, e.g., human motion recognition, using the data wirelessly sensed by the ISAC devices.

In FEEL, a shared ML model, represented by $\mathbf{w}\in\mathbb{R}^{d}$, is trained collaboratively over all the ISAC devices with the coordination of the edge server. Specifically, we seek to find such a model $\mathbf{w}$ that can minimize the objective function,

where $f_{k}(\mathbf{w};\xi)$ is the local loss function at device $k$; $\xi$ is a random seed whose realization represents a batch of samples.

The FEEL training process proceeds iteratively over multiple communication rounds. In an arbitrary round $r$, mainly five steps are executed as elaborated as follows (see Fig. 2):

1. 1.

   Global model broadcasting: Each ISAC device downloads the global FL model $\mathbf{w}^{(r)}$ from the server via the wireless broadcast channel.

2. 2.

   Data sensing: Each ISAC device switches to the sensing mode, and transmits dedicated FMCW signals (as shown in Fig. 3) for sensing. Then, a batch of sensing data, i.e., $\{(x_{i},y_{i})\}_{i\in\mathcal{D}_{k}^{(r)}}$, can be attained at each ISAC device by processing its received echo signals as mentioned. This batch of data with size $b^{(r)}=\left|\mathcal{D}_{k}^{(r)}\right|$ will be used for local computation in the following Step 3. The batch size $b^{(r)}$ can vary adaptively over different rounds, but keep unchanged within any particular round³³3The size of batch can also vary adaptively among different devices in each communication round to reduce the waiting time [25]. In this work, we focus on adjusting batch size over different communication rounds by designing the sensing strategy to accelerate the convergence speed of FEEL..

3. 3.

   Local model updating for $\tau$ steps: Each device updates its model by running $\tau$ steps of stochastic gradient decent (SGD) from $\mathbf{w}^{(r)}$, i.e.,

   	$\displaystyle\mathbf{w}_{k}^{(r,0)}$	$\displaystyle=\mathbf{w}^{(r)},$
   	$\displaystyle\mathbf{w}_{k}^{(r,\ell+1)}$	$\displaystyle=\mathbf{w}_{k}^{(r,\ell)}-\eta\mathbf{g}_{k}(\mathbf{w}_{k}^{(r,\ell)}),\;\ell=0,2,...,\tau-1,\;\forall k,$

   where $\eta$ is the learning rate; $\mathbf{g}_{k}(\mathbf{w}_{k}^{(r,\ell)})=\frac{1}{b^{(r)}}\sum_{i\in\mathcal{D}_{k}^{(r)}}\nabla f(\mathbf{w}_{k}^{(r,\ell)};x_{i},y_{i})$ is the stochastic gradient.

4. 4.

   Local model uploading: Each ISAC device switches to communication mode and uploads its local model after $\tau$ local updates, i.e., $\mathbf{w}_{k}^{(r,\tau)}$, to the server via the uplink wireless channel.

5. 5.

   Global aggregation: Once the server receives the models from all the devices, it aggregates them to obtain a new global ML model, i.e.,

   $$\mathbf{w}^{(r+1)}=\frac{1}{K}\sum_{k\in\mathcal{K}}\mathbf{w}_{k}^{(r,\tau)}.$$

We assume that each device occupies a non-overlapping communication frequency subcarrier, and the transmissions of the devices are interference-free from each other. The received baseband signal from device $k$ at time $t$ is given by

where $h_{k}(t)\in\mathbb{C}$ denotes the channel from device $k$ to the server, $p_{{\sf c},k}(t)$ is the communication transmit power, $c_{k}(t)$ is the communication transmit signal, and $n_{k}(t)$ is additive Gaussian white noise. Since the latency for transmitting a model to the server is typically much longer than the channel coherence time, we assume that the communication channels from the devices to the server are fast Rayleigh fading channel [26]. Specifically, the channel propagation coefficient between the server and device $k$ is generally modeled as⁴⁴4The time index $t$ will be omitted for simplicity hereafter, unless necessary. $h_{k}=\sqrt{\phi_{k}}\overline{h}_{k}$; here, $\phi_{k}$ describes the large-scale propagation effects, including path loss and shadowing effect, and $\overline{h}_{k}$ represents the small-scale fading modeled as independent and identically distributed (i.i.d.) circularly symmetric complex Gaussian (CSCG) random variables with zero mean and unit variance, i.e., $\overline{h}_{k}\sim\mathcal{CN}(0,1)$. We assume that the large-scale propagation coefficient $\phi_{k}$ remains unchanged during the whole uplink transmission, while the small-scale fading $\overline{h}_{k}$ varies from one coherence time block to another. Moreover, we assume that the channel coefficients are only known at the server by channel estimation. The ergodic capacity, instead of instantaneous capacity, of device $k$, is used to quantify the capacity of the fast Rayleigh fading channel between device $k$ and the server [26], which is represented as a function of $p_{{\sf c},k}$ :

where $B_{0}$ is the bandwidth of each subcarrier, and $N_{0}$ is the noise power spectral density.

The FMCW signal for sensing of device $k$ is denoted as $s_{k}(t)$. The bandwidth of sensing signal is denoted as $B_{\sf s}$, and the duration of each chirp is $T_{\sf p}$, as shown in Fig. 3. Moreover, we focus on sensing/recognizing dynamic targets such as moving humans in this work. We consider the scenario that the devices are geometrically separated at different locations with sufficient distance between each other, such that all the devices can sense the environment over the same bandwidth and will not be interfered with each other⁵⁵5This is well justified by the fact that the range of the sensing applications such as human/object recognition is usually small, typically less than 20 meters [20]..

We use a primitive-based method [27] to model the scattering from the entire human body to the device sensing receiver as a linear time varying system. The scattering along $p$-th path is approximated using superposition of the returns from $B$ body primitives as given by (3) at the bottom of next page, where $p_{{\sf s},k}(t)$ is the sensing transmit power of device $k$ at time $t$, $A$ is a constant related to the gain of the antenna, $G_{k,b,p}(t)$ is the complex amplitude proportional to the radar-cross sections (RCS) of the $b$-th primitive at time $t$ along the $p$-th path, $r_{k,b,p}(t)$ is the distance from the $b$-th primitive to the sensing receiver along the $p$-th path, $c=3\times 10^{8}$ $m/s$ is the light speed, and $\bar{y}_{k,p}(t)$ is the scattering along $p$-th path normalized by the square root of the sensing transmit power.

Then, the received signal from the environment at the sensing receiver of device $k$ is given by

where $\overline{y}_{{\sf s},k,1}(t)$ is the normalized scattering along first-order (direct) reflected path, and $\overline{y}_{{\sf s},k,-1}(t)\triangleq\sum_{p=2}^{P}\overline{y}_{{\sf s},k,p}(t)$ is summation of the normalized scattering along higher-order (indirect) reflected paths, and $e_{k}(t)$ is additive term because of the ground clutter and receiver noise. We assume that the additive ground clutter and receiver noise components are zero-mean complex Gaussian distributed [28]. In practice, the ground clutter is small enough to be ignored in a space where no other dynamic objects exist except one moving target.

To get a closed-form expression of $C_{k}(p_{{\sf c},k})$ in (2), it is rewritten as

where $f_{|h_{k}|^{2}}(x)$ and $F_{|h_{k}|^{2}}(x)$ are the probability density function and cumulative distribution function of the random variable $|h_{k}|^{2}$, respectively. It can be verified that $|h_{k}|^{2}$ follows an exponential distribution, i.e., $|h_{k}|^{2}\sim\text{Exp}(1/\phi_{k})$. Hence, we have $F_{|h_{k}|^{2}}(x)=1-e^{-x/\phi_{k}}$. Then, $C_{k}(p_{{\sf c},k})$ can be calculated as

According to [29, Section 8.212], for real number $a$ and $b>0$, $\int_{0}^{+\infty}\frac{e^{-bx}}{a+x}dx=-e^{ab}\text{Ei}(-ab)$, where $\text{Ei}(x)=\int_{-\infty}^{x}\frac{e^{\rho}}{\rho}d\rho$ is the exponential integral function. Then, $C_{k}(p_{{\sf c},k})$ can be rewritten in closed-form as follows:

It can be verified that $C_{k}(p_{{\sf c},k})$ is an increasing function of $p_{{\sf c},k}$.

In this subsection, we analyze the quality of sensing samples for human motion recognition. Micro-Doppler signature is a characteristic of motion, which can be used for recognizing human motions. A common technique for micro-Doppler analysis is the time-frequency representation, such as spectrograms [30]. Thus, the received sensing signal as in (4) needs to be further preprocessed in order to obtain the spectrograms as the sensing data samples. Each spectrogram is generated from a long sequence of the received signal spanning a duration of $T_{0}$, called unit sensing time, which consists of $M$ chirps, as shown in Fig. 3. We assume that the sensing transmit power will remain unchanged during the duration. The sampling rate is denoted as $f_{\sf s}$. The received raw signal as shown in (4) after sampling in $i$-th duration can be reconstructed as a 2D sensing data matrix, i.e., $\mathbf{Y}_{{\sf s},k}[i]\in\mathbb{C}^{f_{\sf s}T_{\sf p}\times M}$, which is the superposition of first-order scattering, higher-order scattering, and the clutter and receiver noise in 2D format as follows:

where $[\mathbf{Y}_{{\sf s},k}[i]]_{\ell,m}=y_{{\sf s},k}((i-1)T_{0}+\ell+mT)$, $[\mathbf{Y}_{{\sf s},k,1}[i]]_{\ell,m}=\bar{y}_{{\sf s},k,1}((i-1)T_{0}+\ell+mT)$, $[\mathbf{Y}_{{\sf s},k,-1}[i]]_{\ell,m}=\bar{y}_{{\sf s},k,-1}((i-1)T_{0}+\ell+mT)$, $[\mathbf{E}_{k}[i]]_{\ell,m}=e_{k}((i-1)T_{0}+\ell+mT)$, $\ell\in[0,f_{\sf s}T_{\sf p}-1]$ represents the fast time index, and $m\in[0,M-1]$ represents the slow time index.

As shown in Fig. 4, similar to [23], we extract the useful first-order scattering⁶⁶6The higher-order scattering can be utilized as a diverse source to improve the sensing performance [31], which, however, makes the processing more complicated. In this work, the high-order scattering signals are treated as interference to the first-order scattering when generating the spectrograms. from $\mathbf{Y}_{{\sf s},k}[i]$ via singular value decomposition (SVD) based filter. Suppose that the SVD of $\mathbf{Y}_{{\sf s},k}$ is denoted as⁷⁷7The index for each duration is omitted for simplicity, since the signal at each duration will be preprocessed in the same way. $\mathbf{Y}_{{\sf s},k}=\mathbf{U}_{k}\mathbf{\Sigma}_{k}\mathbf{V}_{k}^{\ast}$ where $\mathbf{U}_{k}\in\mathbb{C}^{f_{\sf s}T_{\sf p}\times f_{\sf s}T_{\sf p}}$ and $\mathbf{V}_{k}\in\mathbb{C}^{M\times M}$ are complex unitary matrices, and $\mathbf{\Sigma}_{k}\in\mathbb{C}^{f_{\sf s}T_{\sf p}\times M}$ is rectangular diagonal matrix with singular values on the diagonal. The signal after filtering is $\mathbf{Y}_{{\sf F},k}=[\mathbf{U}_{k}]_{:,r_{a}:r_{b}}[\mathbf{\Sigma}_{k}]_{r_{a}:r_{b},:}\mathbf{V}_{k}^{\ast}$, where $r_{a}$ and $r_{b}$ are pre-determined hyperparameters. It is equivalent to write $\mathbf{Y}_{{\sf F},k}$ as

where $\tilde{\mathbf{U}}_{k}=\left([\mathbf{U}_{k}]_{:,r_{a}:r_{b}}\right)\left([\mathbf{U}_{k}]_{:,r_{a}:r_{b}}\right)^{\ast}$ and $\tilde{\mathbf{V}}_{k}=\left([\mathbf{V}_{k}]_{:,r_{a}:r_{b}}\right)\left([\mathbf{V}_{k}]_{:,r_{a}:r_{b}}\right)^{\ast}$. Denote $\tilde{\mathbf{Y}}_{{\sf s},k,1}=\tilde{\mathbf{U}}_{k}\mathbf{Y}_{{\sf s},k,1}\tilde{\mathbf{V}}_{k}^{\ast}$, $\tilde{\mathbf{Y}}_{{\sf s},k,-1}=\tilde{\mathbf{U}}_{k}\mathbf{Y}_{{\sf s},k,-1}\tilde{\mathbf{V}}_{k}^{\ast}$, and $\tilde{\mathbf{E}}_{k}=\tilde{\mathbf{U}}_{k}\mathbf{E}\tilde{\mathbf{V}}_{k}^{\ast}$. Then, $\mathbf{Y}_{{\sf F},k}$ is rewritten as

Next, short-time Fourier transform (STFT) with a sliding window function $\omega[n]$ of length $W$ is adopted to generate the range-Doppler-time (RDT) cube in order to obtain time-frequency features of $\mathbf{Y}_{{\sf F},k}$. The RDT cube of $\mathbf{Y}_{{\sf F},k}$ after STFT is given by

where $Q$ is the number of overlapping points; $f\in[0,W-1]$ and $m\in\left[1,\frac{M-Q}{W-Q}\right]$ are the frequency and temporal shift indexes, respectively. Then, we non-coherently integrate $S_{k}[\ell,f,m]$ within all the range bins and obtain the integrated STFT of $\mathbf{Y}_{{\sf F},k}$ as

Due to the linearity of STFT and summation operation, $\overline{S}_{k}[f,m]$ can be given by

where $\overline{S}_{k,1}[f,m]$, $\overline{S}_{k,-1}[f,m]$, and $\overline{S}_{e,k}[f,m]$ are integrated STFT of $\tilde{\mathbf{Y}}_{{\sf s},k,1}$, $\tilde{\mathbf{Y}}_{{\sf s},k,-1}$, and $\tilde{\mathbf{E}}_{k}$, respectively. The first-order scattering related term $\sqrt{p_{{\sf s},k}}\overline{S}_{k,1}[f,m]$ represents useful information in the spectrogram; the higher-order scattering related term $\sqrt{p_{{\sf s},k}}\overline{S}_{k,-1}[f,m]$ and the clutter and noise related term $\overline{S}_{e,k}[f,m]$ corrupt the spectrogram as interference.

The latency for each device in a communication round comprises four parts: the global model downloading time, sensing time, local computation time, and local model uploading time, as shown in Fig. 6. Since the server broadcasts the same global model to all the devices over the entire frequency band, and the server usually has greater transmit power than the devices, we can ignore the global downloading time in a communication round. In the next, we formulate the other three parts.

The energy consumption of each device comes from three parts: energy for sensing, energy for local computation, and energy for local model uploading, which are formulated one by one as follows:

Limited by the transmitter, the communication transmit power should be constrained by

where $P_{{\sf c},k}^{\max}$ is the peak communication transmit power. Moreover, according to Remark 1, the sensing transmit power should be constrained by

where $P_{{\sf s},k}^{\max}$ is the peak sensing transmit power.

Under the three kinds of constraints above, the problem of accelerating the training process can be formulated as

It is challenging to solve Problem (P1) due to: (i) The loss function $F(\mathbf{w}^{(r)})$ at communication round $r\in\mathcal{R}\triangleq\{1,2,\ldots,R\}$ has no exact analytical expression; (ii) Generally, we need to find an expression of learning error with adaptive batch size over $R$ communication rounds in order to solve Problem (P1), which is also a non-trivial task; (iii) The constraints (C2) and (C3) are non-convex. Moreover, although $C_{k}(p_{{\sf c},k})$ has a closed-form expression in (5), it involves special function which is non-trivial for analysis. In this work, we will derive an upper bound of the learning error for tractability, instead of the exact loss function in Problem (P1), which is beneficial for jointly allocating SC² resource.

To facilitate the analysis, the following commonly adopted assumptions on local loss functions are made [34, 35, 36, 37].

Under the above assumptions, the following theorem gives an upper bound on the average-squared gradient norm of FL with fixed batch size.

In this work, we consider adaptive batch size $b^{(r)}$ over different communication rounds, instead of fixed batch size. Thus, Lemma 1 cannot be directly applied to solve Problem (P1). Nevertheless, Lemma 1 is helpful for splitting Problem (P1) as shown in the following subsection.

This subsection shows that Problem (P1) can be split into two subproblems equivalently to make Problem (P1) tractable.

First, substituting (7), (8), and (9) into (C2), it follows that

After proper manipulation, we can rewrite (14) as

where $b_{\sf sum}=\sum_{r=1}^{R}b^{(r)}$. Similarly, we substitute (10), (11), and (12) into (C3) and can have that

Therefore, the original Problem (P1) can be rewritten as

Before we split Problem (P2), the following corollary is given to unveil how $b_{\sf sum}$ can affect the convergence speed.

Intuitively, $b_{\sf sum}$ is the summation of the batch sizes at all the communication rounds for each device, and also the number of data samples generated/sensed by each device in the training process. From Corollary 1, the devices needs to generate as many data samples as possible to accelerate the convergence speed. However, on the other hand, generating more data samples means greater consumption of the time and energy resources. Since the devices are time and energy constrained, the total number of generated data samples is limited.

Obviously, $b_{\sf sum}$ only depends on the constraints in Problem (P2). From Corollary 1, we need to find maximum $b_{\sf sum}$ under the constraints in Problem (P2) in order to maximize the convergence speed. Therefore, Problem (P2) can be equivalently solved by: (i) first optimizing $\{p_{{\sf c},k}\}$, $\{p_{{\sf s},k}\}$, and $\{T_{{\sf cm},k}\}$ to maximize $b_{\sf sum}$ under the constraints in Problem (P2); (ii) then optimizing $\{b^{(r)}\}$ given maximum $b_{\sf sum}$ to maximize the convergence speed.

In the following, we solve the first subproblem above to maximize the total sensed data samples, i.e., $b_{\sf sum}$, subject to all the constraints in Problem (P2), which is formulated as

Problem (P3) is challenging to solve due to: (i) Constraints (C1), (15), and (16) are non-convex; (ii) Constraints (C1), (15), and (16) involves $C_{k}(p_{{\sf c},k})$ in (5), which has complex special function. It is generally impossible to solve Problem (P3) by using standard techniques for convex optimization. In the following, we first give the optimal sensing transmit power $\{p_{{\sf s},k}\}$, and then simplify Problem (P3) such that we can use grid search efficiently to find the optimal solutions.

From Constraint (16) we have that

which gives an upper bound of $b_{\sf sum}$. Moreover, this upper bound is decreasing with $p_{{\sf s},k}$. Combined with Constraint (C5), we can obtain that the optimal sensing transmit power $p_{{\sf s},k}^{\ast}$ in Problem (P3) is $p_{{\sf s},k}^{\ast}=P_{{\sf s},k}^{\min}$.

Then, Constraint (16) in Problem (P3) becomes

Constraint (15) can also be transformed into

Then, (17) and (18) can be combined as

Then, substituting (20) into (19) yields

where

which is a function of $p_{{\sf c},k}$. With the above results, Problem (P3) reduces to a single variable optimization problem regarding to $\{p_{{\sf c},k}\}$ as follows

Since $\{p_{{\sf c},k}\}$ are independent with each other for $k\in\mathcal{K}$, it can be verified that the optimal communication transmit power is given by

It is still challenging to obtain $p_{{\sf c},k}^{\ast}$ from (23) directly, since $\Phi_{k}(p_{{\sf c},k})$ is still complicated. In this work, we tend to find $p_{{\sf c},k}^{\ast}$ by grid search in $\left[0,P_{{\sf c},k}^{\max}\right]$. After we have $\{p_{{\sf c},k}\}$, the optimal $b_{\sf sum}$ is obtained by substituting $\{p_{{\sf c},k}^{\ast}\}$ into (19), i.e.,

It follows from (20) that the optimal communication time is given by

After obtaining the maximum number of total sensed data samples, i.e., $b_{\sf sum}^{\ast}$, we tend to solve the second subproblem, which optimizes $\{b^{(r)}\}$ under given $b_{\sf sum}^{\ast}$ to maximize the convergence speed.

For given $b_{\sf sum}^{\ast}$, Problem (P2) becomes