Stochastic Resource Allocation for Semantic Communication-aided Virtual Transportation Networks in the Metaverse

Before the data are transmitted to the VSPs, the VSPs have to subscribe to the resources in advance to secure them as separate entities (edge devices) own these resources. There are two types of subscription plans, i.e., reservation and on-demand plans. The reservation plan is treated as the long-term plan, which allows the VSPs to maintain a long-term collaboration with the edge devices. In contrast, the on-demand plan is a short-term plan. It is used only when the edge device’s service is needed temporarily, e.g., when the reservation plan is insufficient to meet the semantic data demand.

To enjoy the reservation plan, the VSPs have to pay a membership fee (monthly) to the owner of the edge devices. This entitles the VSP to a lower costing bundle. Note that each bundle supports $n$ transmissions for the VSP, i.e., one transmission includes that the edge device performs sensor data collection, semantic extraction, and wired or wireless transfer. Let $C_{e}^{(r,memb)}$ denote the membership cost of each edge device and $C_{e}^{(r,trans)}$ represent the reservation cost for each bundle charged by edge device $e$. However, each VSP that is providing services in the Metaverse has a different demand from its users, and the demand is not known when the reservation of the edge devices is made. Consider the situation that, autonomous vehicles (AVs) are expensive and unsafe to train on the physical road involving other vehicles and pedestrians. Instead, the virtual world can be a good medium to train AVs in realistic scenarios. For example, the company Oxbotica uses the Metaverse to improve its AVs’ object detection algorithms [9]. As an illustration, the VSPs serving the users may request 100 semantic road data to train the AV further. However, if the weather changes rapidly, the VSP will require more semantic data that corresponds to the unforeseen weather scenario, e.g., 400 transmissions instead. Therefore, in this case, an on-demand plan is needed to accommodate the shortfall. The cost of each on-demand transmission is denoted as $C_{e}^{(o,trans)}$.

With the uncertainty mentioned above, the demand for VSPs is not always fixed. Let $\lambda_{i}$ denote the demand scenario $i$ of all the VSPs. The set of scenarios is denoted by $\Omega$, i.e., $\lambda_{i}\in\Omega$. Let $P(\lambda_{i})$ denote the probability that scenario $\lambda_{i}\in\Omega$ is realized, where $P(\lambda_{i})$ can be obtained from the historical records [7]. The uncertainty of demands is expressed as follows:

$\lambda_{i}=\begin{bmatrix}(F_{1},\bar{F}_{1},\tilde{F}_{1}),\ldots,(F_{W},\bar{F}_{W},\tilde{F}_{W})\end{bmatrix}$,

We assume that each edge device is allocated with an orthogonal spectrum resource block to avoid the co-interference among the edge devices [10]. Let $r_{e}$ denote the uplink data transmission rate from edge device $e$ to the BS under its coverage. Then, the transmission time taken $t_{e}$ is defined as follows [11]:

where $q_{e}$ is the size of the transmitted data. Let $w$ denote the transmit power used by edge device $e$ in the uplink transmission, and the energy consumption of the transmission is represented as follows [11]:

Therefore, using the energy consumption model, $C_{e}^{(r,trans)}~{}=~{}\frac{\hat{n}w\bar{q}}{r_{e}}\alpha_{1}$ and $C_{e}^{(o,trans)}=\frac{w\bar{q}}{r_{e}}\alpha_{2}$. For simplicity, the average transmitted data size $\bar{q}$ is used, and it is obtained from past historical records. $\hat{n}$ is the number of semantic data transmissions. $\alpha_{1}$ and $\alpha_{2}$ are the cost coefficient associated with energy consumption, where $\alpha_{2}>\alpha_{1}$ since the on-demand plan is typically more expensive than the reservation plan.

Using the plans, the VSPs can obtain semantic data captured by the edge devices. However, it is difficult to identify which edge device produces images that are important or relevant to the interests of the VSPs. This paper adopts a pre-trained machine learning model, you only look once (YOLO) from [12]. With the help of YOLO, objects (semantic data) and their corresponding categories can be extracted from the respective images. YOLO provides real-time detection with relatively high accuracy. When the category is within the interest of the VSPs, the semantic data, which is the snapshot of the object, can be transmitted.

Once the categories are generated from the images, the VSPs cannot identify which edge device to subscribe to as the VSPs do not know which semantic data is relevant to their interest. It is not practical for the VSPs to search manually through the categories. Therefore, we propose to use category similarity for the VSPs to subscribe to the edge device that produces the best semantic data that meets the interest of the VSPs. However, in different contexts, the same word might have multiple definitions. For example, “wind” can mean the current of air or the action turn. The traditional method, such as word2vec cannot recognise polysemy. The issue arises when the same word cannot be represented by the same numerical vector in different contexts. One of the solutions is to use BERT [13]. BERT is a powerful pre-trained machine learning model that has been trained by billions of sentences for extracting semantic information. It is used to convert multiple categories and convert them into vectors according to different contexts. In this paper, we use cosine similarity [13] to measure the similarity between the two sentences. Specifically,

where A is the vectorized of $F_{w}$ and B is the vectorized output generated from BERT. The category similarity defined in( 3) is a number between 0 and 1, which indicates how similar A is to B, with 1 representing the highest similarity and 0 representing no similarity. The average similarity of VSP $w$’s demands and edge device $e$’s data types is represented by $S_{w,e}$ and can be calculated from $S_{w,e}=\frac{\sum_{i=1}^{y}\text{match}(\textbf{A},\textbf{B}_{i})}{y}$, where $y$ is the total number of images in edge device $e$. As a result, the VSPs can subscribe to receive the semantic data from the edge device that has the highest similarity score.

A virtual transportation network is used as a case study to illustrate the system model. In this example, there are two VSPs, VSP 1 and VSP 2. VSP 1 is a company to provide the service of the AV, while VSP 2 is a bus company. AVs are dangerous to travel on the road when the machine learning model is not fully trained. However, the vehicles are unable to train when they are not allowed to travel on the road. Therefore, one solution for VSP 1 is to set up a simulated environment in the Metaverse, which digitises the physical road using real data as input. VSP 1 can then test the AVs’ detection systems in the Metaverse and update the machine learning model depending on users’ demands in the physical world.

Similarly, it is also dangerous for a pedestrian when VSP 2 conducts their on-the-job training for new bus drivers as the new drivers are unfamiliar with the traffic conditions or inexperienced. Therefore, virtual on-the-job training can be conducted in the Metaverse. VSP 2 can replicate the transportation networks in the physical world into the Metaverse. However, in order to support the Metaverse, VSPs 1 and 2 require a tremendous amount of data from the edge devices in the physical world to support their Metaverse. For example, VSP 1 may have the interest of $F_{1}=$ vehicles on the road. Different vehicle images can be used to constantly update the database to improve the Metaverse for autonomous vehicles. Meanwhile, VSP 2 may have the interest of $F_{2}=$ images related to the buses and traffic lights so that VSP 2 can monitor the physical driving condition of the bus driver and improve their virtual on-the-job training procedure.

In order to provide a more realistic or practical scenario, we deploy three different smartphones to act as edge devices at three different locations in Singapore. The locations are shown in Fig 2. The VSPs do not know whether the images captured by smartphones are related to their interest. Therefore, before the VSPs subscribe to which smartphone to use, each smartphone owner provides $\bar{n}$ data for the VSPs to study the historical records. From the historical captured images, the VSPs can extract the category and also the corresponding snapshots (semantic data). An average similarity score can be obtained from the historical category generated by the respective smartphones used to compare with the interest of the VSPs. Figure 2 is used to illustrate the case study. The average similarity score helps the VSPs choose the optimal smartphone to subscribe, for example, using the average similarity score obtained in Fig 2. When the threshold is 100% $\tilde{F}_{w}=1$, $n=120$ and the number of semantic data transmissions required is $\bar{F}_{w}=100$, it is definitely cheaper for the VSP 1 to subscribe smartphone 3. As out of the data transmitted, 83% of the data is related to the interest of VSP 1. Smartphone 3 can meet the demand of VSP 1 by using only 1 bundle, $1\times n\times 83\%\geq\bar{F}_{w}\times\tilde{F}_{w}$. The snapshots of the objects (semantic data) are transmitted to the respective VSPs once the smartphone is subscribed. However, each VSP experiences different user demands, making it difficult for VSPs to subscribe for the optimal plan. Therefore, this paper uses a stochastic allocation scheme to optimize the resources by considering the demand uncertainties.

The VSPs can subscribe to the optimal edge devices and purchase the optimal number of bundles for semantic data transmission by using the reservation plan when the demand is precisely known. Therefore, the on-demand plan is not required. In total, there are two decision variables.

• •

  $m_{w,e}^{(\mathrm{r})}\in\{0,1\}$ indicates whether VSP $w$ pays the membership cost to the owner of the edge device $e$.

• •

  $\bar{m}_{w,e}^{(\mathrm{r})}\in\{0,1,\ldots\}$ indicates the number of bundles purchased by the VSP $w$.

A DIP can be formulated to minimize the total cost of the VSPs as follows:

subject to:

The objective function in (4) is to minimize the total cost due to subscription reservations. $D_{w,e}$ is the actual average similarity score between the interest of the VSP $w$ and the edge device $e$. $\bar{D}_{w}$ is the actual image demand of VSP $w$ and $\tilde{D}_{w}$ is the actual acceptable threshold of VSP $w$. The constraint in (5) ensures that the VSP has to pay the membership fee to the edge device owner before the VSP can purchase any bundle from the respective edge device. (6) ensures that the demand is met. For example, when the number of edge devices is 1, $e=1$, $n=200,D_{w,1}=0.8$, $\bar{D}_{w}=200$, and $\tilde{D}_{w}=1$. It means that only $80\%$ of the data that are captured by the edge device $1$ is relevant to the interest of VSP $w$. VSP $w$ faces a shortfall of $20\%$, and the acceptable threshold of $w$ is 100%. As a result, instead of 1 bundle, VSP $w$ has to purchase two bundles $m_{w,1}^{(\mathrm{r})}=2$ to meet the demand of 200 semantic data transmissions. (7) indicates that $m_{w,e}^{(\mathrm{r})}$ is a binary variable. (8) indicates that $\bar{m}_{w,e}^{(\mathrm{r})}$ is a non-negative integer.

If the demands of the VSPs are not known, the DIP formulated in (4) - (8) is no longer applicable. Therefore, SIP with a two-stage recourse is developed. This section introduces the SIP to minimize the total cost of the network by optimizing the number of edge devices to subscribe to and the number of semantic data transmissions to subscribe. The first stage consists of all decisions that must be selected before the demands are realized and observed. The VSPs have to subscribe to the edge device and purchase the corresponding bundle before observing the demands. In the second stage, decisions are allowed to adapt to the demand observed. After the demand is observed, the VSPs have to pay for the additional images needed if the number of images reserved in the first stage is lesser than the demand.

Besides the two decision variables listed in Section III-A, there is one more decision variable in the SIP formulation. $m_{w,e}^{(\mathrm{o})}(\lambda_{i})\in\{0,1,\ldots\}$ indicates the number of semantic data transmissions that VSP $w$ is requested on-demand from the edge device $e$ in scenario $\lambda_{i}$.

The objective function given in (III-B) and (10) is to minimize the cost of resource allocation. The expressions in (III-B) and (10) represent the first- and second-stage SIP, respectively. The SIP formulation can be expressed as follows:

where

subject to: (5), (7)-(8)

(11) is similar to (6), it is to ensure the demand is met. (12) indicates that $m_{w,e}^{(\mathrm{o})}(\lambda_{i})$ is non-negative integer.