Learning Optimal Fronthauling and Decentralized Edge Computation in
Fog Radio Access Networks

As shown in Fig. 2(a), EN $i$ first sends the information regarding its local observation $\mathbf{a}_{i}$ to the cloud using the uplink fronthaul link assigned with $M_{i0}$ RBs. A straightforward transmission of the $A_{i}$-dimensional raw data $\mathbf{a}_{i}$ would not be possible when we have insufficient fronthaul resources as $M_{i0}\leq A_{i}$. Thus, EN $i$ needs to identify a low-dimensional representation of $\mathbf{a}_{i}$ without no direct interactions with other ENs. This can be viewed as decentralized edge compression steps. The resulting representation $\mathbf{m}_{i0}\in\mathbb{R}^{M_{i0}}$ of length $M_{i0}$ is referred to as an uplink message that carries the local knowledge of EN $i$ to the cloud via $M_{i0}$ fronthaul RBs. Let $\mathcal{M}_{i}(\cdot)$ be a computational inference performing the uplink message generation of EN $i$, i.e.,

In (1), only the local observation $\mathbf{a}_{i}$ is accepted as an input for characterizing fully decentralized processing. As discussed in Section IV, the inference $\mathcal{M}_{i}(\cdot)$ is modeled by a DNN to be optimized for maximizing the utility.

Practical fronthaul links are interrupted with channel impairments such as the noise, and thus the cloud would get the noisy observation of the uplink messages. To capture this, we introduce a channel transfer function $h_{i0}(\cdot)$ for the uplink fronthaul link from EN $i$ to the cloud which can include any channel imperfection encountered in the uplink communication. Then, the received signal at the cloud $\mathbf{y}_{0}$ depends on all the noisy uplink messages $h_{i0}(\mathbf{m}_{i0})$, $\forall i$. It is written by

where the function $u(\cdot)$ defined over the set of the noisy messages $\{h_{i0}(\mathbf{m}_{i0}):\forall i\}$ describes an uplink transmission strategy of the ENs. The choice of $u(\cdot)$ relies on the fronthaul resource sharing policy. For instance, if each EN occupies distinct fronthaul RBs, $u(\cdot)$ is simply given by the concatenation operation. On the other hand, $u(\cdot)$ becomes the summation when all ENs share the entire uplink fronthaul RBs. The dimension of $\mathbf{y}_{0}$ depends on the number of the uplink fronthaul RBs $M_{i0}$ and the uplink signaling strategy $u(\cdot)$. These are specified in Section V.

From (2), we can observe that the received signal $\mathbf{y}_{0}$ conveys distorted information of the global observation $\mathbf{a}=\{\mathbf{a}_{i}:\forall i\}$ with the piecewise edge processing $\mathcal{M}_{i}(\cdot)$. A standard approach to process with $\mathbf{y}_{0}$ is to decompose the computations of the cloud into the following subsequent steps. The cloud first recovers the global state $\mathbf{a}$ from the received signal $\mathbf{y}_{0}$. Then, the solution to (P1) is determined by centralized cloud computing strategies. The resulting solution $\mathbf{x}_{i}\in\mathbb{R}^{X_{i}}$ is sent back to EN $i$ via the downlink fronthaul links with $M_{0i}$ RBs. To handle the practical case with $M_{0i}\leq X_{i}$, $\mathbf{x}_{i}$ is encoded into a downlink message $\mathbf{m}_{0i}\in\mathbb{R}^{M_{0i}}$ whose dimension $M_{0i}$ is fit to the number of the downlink fronthaul RBs $M_{0i}$ assigned to EN $i$. As illustrated in Fig. 2(b), we integrate such cascaded procedures into a single computation inference $\mathcal{M}_{0}(\cdot)$ that creates a set of the downlink messages $\{\mathbf{m}_{0i}:\forall i\}$ from $\mathbf{y}_{0}$. This can be written by

It is inferred from (3) that the downlink message $\mathbf{m}_{0i}$ encapsulates the local observations of other nodes $\mathbf{a}_{j}$, $\forall j\neq i$, as well as an intermediate decision taken at the cloud. The inference $\mathcal{M}_{0}(\cdot)$ is also modeled by a DNN whose parameters are determined to maximize the utility function.

The distributed decision process shown in Fig. 2(c) is described. The cloud broadcasts the downlink messages to EN $i$ with a pre-designed downlink signaling strategy denoted by $d_{i}(\cdot)$. Similar to the uplink signaling strategy $u(\cdot)$ in (2), $d_{i}(\cdot)$ is defined over the set of the downlink messages $\{\mathbf{m}_{0i}:\forall i\}$ and becomes a design factor to be specified in Section V. The downlink signal intended to EN $i$, which is denoted by $\mathbf{d}_{i}$, can be written by

Defining $h_{0i}(\cdot)$ as the downlink fronthaul transfer function from the cloud to EN $i$, the received signal $\mathbf{y}_{i}$ at EN $i$ is given by

The dimensions of $\mathbf{d}_{i}$ and $\mathbf{y}_{i}$ rely on the message broadcasting strategy $d_{i}(\cdot)$ to be designed in Section V. Combining (1), (3), and (5), we can see that the received message $\mathbf{y}_{i}$ of EN $i$ contains the local statistics of all the ENs. This implies that all the sufficient, but possibly corrupted, information for solving (P1) is now available at each EN. Thereby, the solution $\mathbf{x}_{i}$ of EN $i$ can be attained individually by means of a node-centric decision inference $\mathcal{X}_{i}(\cdot)$. The proposed solution computation rule at EN $i$ is expressed as

We use the local observation $\mathbf{a}_{i}$ as the side information to refine the received signal $\mathbf{y}_{i}$ dedicated to EN $i$. This additional input forms a residual shortcut which leads to an efficient training strategy of very deep networks [25].

Algorithm 1 summarizes the inference of the CECIL framework. The uplink messages generated at the ENs are first transmitted to the cloud. Receiving the noisy signal $\mathbf{y}_{0}$, the centralized cloud computing yields the downlink messages to be broadcasted to the ENs. The decision $\mathbf{x}_{i}$ is then taken at each EN $i$ individually. The proposed inference relies only on locally observable information, i.e., local measurement $\mathbf{a}_{i}$ and the received messages, but not on instantaneous states of other network entities. As a result, Algorithm 1 can be implemented in a distributed manner with optimized $\mathcal{M}_{i}(\cdot)$ and $\mathcal{X}_{i}(\cdot)$.

Fig. 3 illustrates the CECIL-based F-RAN systems where the computations of the ENs and cloud are carried out by the DNNs in (8)-(10). The forward pass computations of the CECIL are provided in Algorithm 1. Plugging (8)-(10) to (P2) results in

where $\boldsymbol{\Theta}$ accounts for the set of learnable parameters of the DNNs in (8)-(10) defined as

To remove the constraint of (P2), the output activation of $\mathcal{F}_{Q_{X_{i}}}(\cdot;\theta_{X_{i}})$ can be designed as the projection operator $\arg\min_{\mathbf{v}\in\mathcal{D}_{i}}\|\mathbf{u}-\mathbf{v}\|$ for a layer input $\mathbf{u}$. For the convex feasibility set $\mathcal{D}_{i}$, this projection activation is given by a convex quadratic program (QP) whose gradient-based training rules can be obtained with the backpropagation algorithm [27]. The nonconvex projection problem can be tackled by the successive convex approximation mechanism [28] by solving a series of approximated convex QPs. The gradients of such an iterative procedure can be obtained by integrating the gradients of approximated convex QPs. As a consequent, (P2) is readily solved by the gradient descent method and its variants for stochastic optimizations, e.g., the Adam algorithm [29]. We adopt the mini-batch stochastic gradient descent (SGD) method [30] where the expectations over the distribution of $\mathbf{a}$ are estimated as the sample mean evaluated on the mini-batch sets $\mathcal{A}\triangleq\{\mathbf{a}\}$. The SGD update at the $t$-th training epoch is given by

where $q^{(t)}$ indicates a variable $q$ attained at the $t$-th epoch, $\alpha>0$ is a learning rate, and $\nabla_{q}$ denotes the gradient operator with respect to $q$. The sample gradient $\nabla_{\boldsymbol{\Theta}}f(\mathbf{a},\{\mathcal{F}_{Q_{X_{i}}}(\mathbf{a}_{i}\oplus\mathbf{y}_{i};\theta_{X_{i}}:\forall i\})$ can be numerically calculated by the backpropagation algorithm [30], provided that the gradients of the channels $h_{i0}(\cdot)$ and $h_{0i}(\cdot)$ as well as the transmission strategies $u(\cdot)$ and $d_{i}(\cdot)$ are available.

The full knowledge of the global observation $\mathbf{a}$ is required for computing the gradient of the utility function. This can be achieved by the centralized training procedure in an offline domain before real-time optimization inferences [16, 17, 18, 19]. To this end, we can collect training samples, i.e., a set of the local observation vectors $\mathbf{a}_{i}$, from the ENs in advance. No labels such as the information regarding the optimal solution to (P1) are needed in the training. Thus, the proposed training strategy (13) is performed in a fully unsupervised manner. Once the parameter set $\boldsymbol{\Theta}$ is determined, they are readily implemented at the cloud and ENs. As discussed, the forward pass in Algorithm 1 can be carried out only with locally measurable statistics, thereby leading to the distributed realization of the online computations (8)-(10). Compared to existing decentralized F-RAN optimization algorithms [9, 10] that require iterative procedures, the proposed CECIL does not need any repetitions in the real-time inference step. Hence, the proposed approach can save both the fronthaul signaling and computation overheads.

We first develop an OMA method where distinct fronthaul resources are assigned to each of uplink and downlink messages to avoid inter-message interferences. The uplink messages $\mathbf{m}_{i0}\in\mathbb{R}^{M_{i0}}$ for $i=1,\cdots,N$ occupy $N$ bundles of the fronthaul RBs where the $i$-th resource bundle containing $M_{i0}$ RBs is dedicated to the uplink message transmission of EN $i$. In this setup, the uplink signaling strategy $u(\cdot)$ in (2) becomes the concatenation operation. Then, the received signals at the cloud (2) is rewritten by

where $\bigoplus_{i=1}^{N}\mathbf{q}_{i}\triangleq[\mathbf{q}_{1}^{T},\cdots,\mathbf{q}_{N}^{T}]^{T}$ defines the concatenation of $N$ vectors $\mathbf{q}_{i}$ for $i=1,\cdots,N$. The dimension of $\mathbf{y}_{0}^{\text{OMA}}$ becomes $M_{U}\triangleq\sum_{i=1}^{N}M_{i0}$ where $M_{U}$ indicates the total number of the uplink fronthaul RBs.

In the downlink, $\mathbf{m}_{0i}\in\mathbb{R}^{M_{0i}}$ is sent on $N$ orthogonal downlink fronthaul links each having $M_{0i}$ RBs. Hence, the downlink signaling strategy $d_{i}(\cdot)$ in (4) can be specified as a masking operation extracting $\mathbf{m}_{0i}$ from the downlink message set $\{\mathbf{m}_{0j}:\forall j\}$, i.e., $\mathbf{d}_{i}=\mathbf{m}_{0i}$. Combining this with $\mathcal{M}_{0}(\cdot)$ in (3), the downlink message generation of the OMA system can be refined as the procedure that creates the concatenation of $N$ downlink messages. It follows

The downlink message $\mathbf{m}_{0i}$ is then received by EN $i$ through the corresponding downlink fronthaul channel $h_{0i}(\cdot)$. Hence, we refine the received signal at EN $i$ in (5) as

Since $M_{0i}$ RBs are allocated for the transmission of $\mathbf{m}_{0i}$, the length of $\mathbf{y}_{i}^{\text{OMA}}$ is given by $M_{0i}$, resulting in $M_{D}\triangleq\sum_{i=1}^{N}M_{0i}$ downlink fronthaul RBs. Therefore, the total number of the RBs denoted by $M$ is written by $M=M_{U}+M_{D}=\sum_{i=1}^{N}(M_{i0}+M_{0i})$.

The OMA strategy may waste the fronthaul resources for allocating distinct RBs for each EN. To this end, we propose a non-orthogonal message transmission scheme where all ENs share the same fronthaul resources. Provided that $M_{U}$ RBs are assigned for the uplink message transmission, EN $i$ obtains its messages $\mathbf{m}_{i0}$ from (1) by setting $M_{i0}=M_{U}$, i.e., utilizing all uplink fronthaul RBs. Then, the uplink transmission strategy $u(\cdot)$ is obtained as the superposition of all the downlink messages since they are interfere with each other. Therefore, the cloud receives the superposed signal $\mathbf{y}_{0}^{\text{NOMA}}\in\mathbb{R}^{M_{U}}$ of length $M_{U}$ expressed as

In the downlink, the cloud multicasts a common downlink message $\mathbf{m}_{0}\in\mathbb{R}^{M_{D}}$ of length $M_{D}$ to all the ENs by leveraging all the available $M_{D}$ downlink fronthaul RBs. Then, the downlink signaling in (2) is simply fixed as $\mathbf{d}_{i}=\mathbf{m}_{0}$, $\forall i$, such that the cloud directly transmits the output of the cloud computation in (18). We thus modify (3) for the NOMA scheme as

Accordingly, the received signal $\mathbf{y}_{i}^{\text{NOMA}}\in\mathbb{R}^{M_{D}}$ of length $M_{D}$ at EN $i$ can be rewritten by

We discuss the effectiveness of the OMA and NOMA schemes for the perfect fronthaul link case, i.e., $h_{i0}(\cdot)$ and $h_{0i}(\cdot)$ are given by the identity functions. The received signals of the OMA and NOMA systems are respectively recast to

which simplifies (15) and (18) as

The imperfection of the wireless fronthauls can be modeled by the random additive noise. We specify the fronthaul channel functions as $h_{i0}(\mathbf{u})=h_{0i}(\mathbf{u})=\mathbf{u}+\boldsymbol{\eta}$ where $\boldsymbol{\eta}$ stands for the noise vector with arbitrary distribution. In the OMA system, the received messages $\mathbf{y}_{0}^{\text{OMA}}$ at the cloud (14) and $\mathbf{y}_{i}^{\text{OMA}}$ at EN $i$ (16) are respectively written by

where $\boldsymbol{\eta}_{i}$ for $i=0,1,\cdots,N$ denotes the noise at node $i$. We obtain similar formulations for the NOMA system as

The noise hinders successful decisions at the ENs, thereby requiring robust message generation strategies both at the cloud and ENs. To this end, we modify the training update in (13) by taking the noise into account. We include numerous realization of the random noise vectors into the training set. A mini-batch set $\mathcal{A}$ becomes a set of tuples $(\mathbf{a},\{\boldsymbol{\eta}_{i}:\forall i\})$ of the global observation $\mathbf{a}$ and a collection of the uplink and downlink noise vectors $\{\boldsymbol{\eta}_{i}:\forall i\}$. The DNN parameter $\boldsymbol{\Theta}$ is then adjusted in the ascent direction of the gradient averaged over the noise distribution. Such a data-driven optimization enables robust design of the CECIL by observing numerous noisy messages (31) and (32) in the training step.

Until Section. V, we assumed lossless fronthaul interactions where each RB can convey a real-valued scalar number without any distortion. In the practical wired fronthaul link setup, however, the resolution of the message would be limited by the fronthaul capacity. To this end, in this subsection, we design a robust training policy of the CECIL for the general case where the fronthaul links are subject to the transmission capacity. The fronthaul channels $h_{i0}(\cdot)$ and $h_{0i}(\cdot)$ can be given as the rounding functions that output the nearest integer of the transmitted messages. In this configuration, only the lossy coordination is allowed to share discrete-valued messages. To accommodate capacity-limited fronthaul links, we present a message quantization process that creates discrete representations of continuous-valued messages. We focus on the quantization of the uplink message $\mathbf{m}_{i0}$, but the proposed techniques are readily applied to the downlink message quantization. Let $\hat{\mathbf{m}}_{i0}\in\mathbb{R}^{M_{i0}}$ be the quantization output of $\mathbf{m}_{i0}$. The capacity of the uplink fronthaul link connecting EN $i$ and the cloud is modeled by a set of integers $C_{il}$, $\forall l=1,\cdots,M_{i0}$, each of which indicates the alphabet size, or equivalently, the modulation level allowed for transferring the $l$-th element $[\mathbf{m}_{i0}]_{l}$. It is expressed as

where $\psi_{C}(\cdot)$ stands for the quantization function with the quantization level $C$. It maps a continuous-valued input into a discrete set $\{0,1,\cdots,C-1\}$. The received signals in (2) and (5) can then be refined as $\mathbf{y}_{0}=u\big{(}\{h_{i0}(\hat{\mathbf{m}}_{i0}):\forall i\}\big{)}$ and $\mathbf{y}_{i}=h_{0i}\big{(}d_{i}(\{\hat{\mathbf{m}}_{0j}:\forall j\})\big{)}$, respectively.

The quantization operator $\psi_{C_{il}}(\cdot)$ is viewed as an activation function that is followed by $\mathcal{M}_{i}(\cdot)$, i.e., the DNN $\mathcal{F}_{Q_{M_{i}}}(\cdot;\theta_{M_{i}})$ in (8). Our target is to design the activation $\psi_{C_{il}}(\cdot)$ such that $\hat{\mathbf{m}}_{i0}$ acts as an accurate estimate of the original message $\mathbf{m}_{i0}$. In this way, the cloud and ENs can successfully recover the original messages through their quantized observations. One naive approach would be to employ the rounding function. However, the simple rounding activation exhibits zero gradient for all input regime, thereby prohibiting the DNN parameters from being optimized using the SGD method in (13). This has been well-known as the vanishing gradient problem where the performance of the DNNs are no longer improved but possibly gets stuck into an unsatisfactory point [30]. In our case, the DNNs $\mathcal{F}_{Q_{M_{i}}}(\cdot;\theta_{M_{i}})$ in (8) and (9) would not be trained properly. To handle this difficulty, a novel quantization method has been provided in [19, 36], but it is only applicable to the special case of $C_{il}=2$.

We propose an integerization technique which is regarded as an extension of the binarization method in [19] for the general case of $C_{il}>2$. The $l$-th element of the continuous-valued message $\mathbf{m}_{i0}$ is assumed to lie in a bounded region $[0,C_{il}-1]$. This can be achieved by applying a bounding activation, e.g., the sigmoid function, to the output layer of the DNN $\mathcal{F}_{Q_{M_{i}}}(\cdot;\theta_{M_{i}})$. The proposed quantization function $\psi_{C_{il}}(\cdot)$ in (33) carries out a randomized rounding operation. It first configures two nearest integers $c-1$ and $c$, $\forall c=1,\cdots,C_{il}-1$, of the input $[\mathbf{m}_{i0}]_{l}$, i.e., $[\mathbf{m}_{i0}]_{l}\in[c-1,c)$, as candidates of the quantization. For notational simplicity, we denote $m\triangleq[\mathbf{m}_{i0}]_{l}$ and $\hat{m}\triangleq[\hat{\mathbf{m}}_{i0}]_{l}$. Provided that $m\in[c-1,c)$, the rounding output $\hat{m}=\psi_{C_{il}}(m)$ can be either $c-1$ or $c$ with probabilities

The probabilities in (34) and (35) can be interpreted as the distances from the continuous input $m$ to the target quantization points $c$ and $c-1$, respectively. The probability $\Pr\{\hat{m}=c|m\in[c-1,c)\}$ increases as $m$ gets closer to $c$, and the resulting quantization $\hat{m}$ is more likely to be $c$.

The proposed quantization activation $\psi_{C_{il}}(m)$ for an input $m\in[0,C_{il}-1)$ is given as

where $\mathds{1}_{Z}\in\{0,1\}$ denotes the indicator function which is $1$ if the condition $Z$ is true and $0$ otherwise. The following proposition states the quality of the quantization $\hat{m}=\psi_{C_{il}}(m)$ in terms of its estimation property for unavailable information $m$.

Proposition 2 reveals that the cloud and ENs can accurately recover the continuous-valued messages by taking expectations over the received quantized messages. This can be realized with numerous quantization samples observed in the training step. Therefore, the DNN at the cloud $\mathcal{F}_{Q_{M_{0}}}(\cdot;\theta_{M_{0}})$ in (9), which processes the quantized uplink messages $\hat{\mathbf{m}}_{i0}$, can be trained to decode the original information $\mathbf{m}_{i0}$ successfully.

Now, we discuss an efficient training strategy of the DNNs implemented with the probabilistic activation (36), which is, in general, has no closed-form expression for the gradient $\nabla_{\boldsymbol{\Theta}}\psi_{C_{il}}(m)$. To address this, the gradient estimation techniques [37, 38, 19, 36] is adopted which approximate an intractable gradient with its average evaluated over any randomized operations. By leveraging Proposition 2, the gradient $\nabla_{\boldsymbol{\Theta}}\psi_{C_{il}}(m)$ can be approximated as