Spatial Channel State Information Prediction with Generative AI: Towards Holographic Communication and Digital Radio Twin

As mentioned earlier, the predominant method for acquiring channel information is pilot-signal-based channel estimation, which can serve as a prediction for only the next few transmissions with a lack of generalization capability. Consequently, many statistical channel models have been proposed based on large amount of field measurement, which can effectively predict signal strength levels based on Tx-Rx distance and empirical parameters for different environments. However, statistical models fall short in providing the certainty and spatial information about its channels, which are crucial for holographic communication.

In contrast, deterministic channel models are grounded in detailed environmental information and EM theory to describe the propagation of EM waves. Among these models, computational electromagnetic methods (CEMs) are the most solid ones, solving Maxwell’s equations with detailed boundary conditions through computational methods like the finite element method and finite-difference time-domain [9]. These methods offer high accuracy and are particularly effective in small-scale environments, but become increasingly challenging and less practical in large-scale environments due to the immense computational demands.

Considering that the average size of objects and environments in large-scale environments is substantially larger than cellular EM wavelength, most space can be regarded as the far-field relative to the Tx, and it is feasible to employ the ray concept from geometric optics as an approximation to the propagation of EM wave [10]. This approximation has led to the widespread adoption of ray tracing methods for radio wave simulation, offering a balance between computational efficiency and accuracy in complex, large-scale environments.

The explosive advancement in AI fields and the decreasing costs of computation and sensing hardware, are bringing us closer to holographic communication with real-time spatial-CSI prediction. Spatial-CSI, which is distinct from traditional CSI due to its detailed attributes of each path as mentioned before, enables the deterministic construction of spatial streams. This means that data streams can be precisely mapped to specific radio beams, which are then finely steered towards valid LoS and NLoS paths. Meanwhile, the receivers adjust their receiving beam patterns to align with these transmitted beams, facilitating interference-free reception. Moreover, as the spatial resolution of the system increases and more RISs are deployed to augment spatial wireless capacity, the benefits gained from holographic communication systems are expected to escalate rapidly. Therefore, the prediction of spatial-CSI, while a stiff and distant challenge, holds immense potential for the advancement of future wireless communication systems.

In urban environments, where changes occur slowly and planned, it is practical to address these long-term elements separately. As depicted in Fig. 2, most of computational tasks of digital twin are implemented within a local smart BS that governs a designated surrounding area. This involves constructing a digital environment, consisting of precise 3D models of the buildings, infrastructures and roads, incorporating visual textures, material segmentation, EM parameters, and textual labels (such as roads, sidewalks, and crossings), similar to the workflow proposed in [5]. Accompanying these long-term elements, middle-term elements, such as weather, temperature, and atmospheric scattering, should be updated regularly, though not in real-time. Upon completion of the basic digital environment, the BS can simulate and maintain a basic 3D radiomap which provides the spatial-CSI between the BS and all the positions in the 3D space. This map serves as an initial channel prediction for further processes.

The scheme of separately addressing short-term elements is based on two key considerations. First, these elements, such as pedestrians and vehicles, are relatively small compared to the overall environment, and do not significantly alter the radio paths. Second, many of these short-term elements are of interest in higher-level applications, such as connected vehicle systems or pedestrian monitoring. At this point, the tasks of calibrating their impact on wireless channels and sensing these targets are organically integrated. For instance, a system may detect the entry of a connected vehicle into its range through variations in the wireless channel, subsequently sensing and integrating this vehicle into the digital twin, thereby updating channel predictions. Consequently, the prevailing challenge is the precise sensing and integration of these elements.

To solve this challenge, intelligent infrastructures, which are directly linked to the BS and equipped with various sensors, can gather a large amount of multi-modal data. Processing and fusion of these multi-modal data have been extensively explored, particularly within the field of CAVs. Beyond the data handling, from a systemic perspective, the sensing targets are classified into connected and unconnected entities. Connected entities, like the CAVs, can be seamlessly integrated into the digital radio twin system through several transmissions, and can continuously report their states, such as velocities, positions, directions and routes, to the digital radio twin system. For unconnected entities, such as bicyclists, the system needs to spend more effort to detect and sense them. This task can be done by collecting the data from sensors and processing this data with the digital radio twin. Besides, those connected entities, which usually possess various sensors, can provide additional data from completely different views.

Image rendering primarily focuses on visible light, with spectrum ranging from 430 THz to 750 THz. During the propagation, these lights interact with objects and give rise to a large number of paths which will collectively influence the imaging fidelity. Rendering high-quality image demands exceptional pixel quantity, and multiple rays per pixel (for instance, rendering at $1920\times 1080$ pixels with 10 rays per pixel requires 20,736,000 rays). Fortunately, human vision is less demanding in terms of spectral and temporal resolution, which only utilizes 24 bits (r, g, b with 8 bit) via a 300 THz bandwidth on each pixel, and 24 frames per second.

In contrast, radio simulation presents a considerable different scenario. Although the radio wavebands are commonly discussed, they actually utilize discrete frequencies evenly distributed within the designated waveband. Contrary to the continuous and mixed spectrum of certain specific color, a significant effort in wireless systems is dedicated to mitigating the interference between these discrete frequencies. The propagation characteristics of radio waves also diverge substantially from light, as material EM properties vary greatly between sub-6G and 500 THz and only reflection and diffraction play pivotal roles. Finally, at the Rx, radio simulation requires extremely high resolution on frequency and time domain but very little in space compared to image rendering.

From this comparison, it is apparent that there is a need to optimize the ray tracing methods for radio simulation, termed as “radio tracing”. The challenges in radio tracing primarily reside in efficiently identifying the geometrical paths, and precisely modeling and calibrating the EM wave propagation process. Unlike image rendering, where the vast majority of emitted rays contribute to the final image quality, in radio simulation, the number of valid paths rarely exceeds 200, suggesting a more targeted approach in identifying these paths. Once the accurate geometrical paths are determined, the next challenge is calculating the precise channel characteristics. Contrary to image rendering, which focuses on the 24-bit color information of each pixel, radio simulation requires accuracy in attenuation, phase shift, delay, and polarization, necessitating considerable effort in more refined modeling method, field measurement, and calibration methods.

In this experiment, we aim to evaluate whether a neural network can effectively learn the correlation between the radio environment (including environmental information, Tx and Rx positions) and the spatial paths. We utilized a generative network to explore this concept, with the complete workflow illustrated in Fig. 3.

For dataset, we employed the synthetic WiSegRT dataset[13], a wireless communication dataset based on realistic, finely modeled indoor environments and ray tracing. This dataset includes ten scenarios, each featuring 15 to 30 Tx locations and correlating to thousands of Rx positions. To simplify the task and facilitate efficient learning, we selected 130,000 Tx-Rx pairs from three indoor scenes and filtered out paths with over two interactions or low channel gain. Then, we constructed the input radio scenes, with orthogonal projection images from six directions $(+x,-x,+y,-y,+z,-z)$. These images were padded to $1024\times 1024$ pixels to contain enough environment information while maintaining the same scale. Thus, the actual input consists of six $1024\times 1024$ RGB images, totaling 18 channels. The output consists of three orthogonal projection images of the paths, sufficient for reconstructing these paths in 3D space. This results in a target of three $1024\times 1024$ gray scale images, totaling three channels. Regarding the division of training, validation, and test sets, considering our goal to validate the feasibility of this concept, we divided them only based on different Tx locations without setting up zero-shot scenes in validation and testing sets. In ideal case, the data should come from hundreds of different environments which are divided into those sets based on environment to test the network’s generalization capability.

In terms of neural network, we used UNet as the architecture [14], which is widely applied in many generative network architectures due to its encoder and decoder structure with skip connections at multiple network depths. This structure allows for feature extraction from the input domain and restoration to the output domain at different spatial scales. In constructing the convolutional blocks, we used ResNet-18 as the backbone for the encoder part. ResNet [15] is an extensively used model for visual information feature extraction, remains a baseline for performance comparisons in many works. The decoder was built using basic deconvolutional layers. Given that the output is gray scale images, binary cross-entropy (BCE) loss function is selected.

The training of the network was executed on a server equipped with dual A6000 GPUs. However, even with combined GPU memory of 96 GB, the image size of $1024\times 1024$ limits the batch size to 32. Given the dataset’s scale, with nearly 5000 batches per epoch, we set an initial learning rate of 0.0001 in the Adam optimization algorithm to avoid the gradient decent falling into a local optimum in the high-dimensional parameter space in early epochs. The subsequent training results have validated the effectiveness of this setting.

Fig. 4 shows both the training and validation loss consistently decreased as the training progressed. Around the 50 epochs, the validation loss began to fluctuate, while the training loss continued to decrease, indicating a potential overfitting. Consequently, we chose the model with the lowest validation loss before the 50th epoch. The average test loss was measured at 0.001183, which is close to the validation loss. In terms of the model’s performance in predicting paths, as depicted in Fig. 5, it is clear that the model has successfully learned the general propagation patterns within the given scenarios. However, there is still a lack of precision in depicting the paths. Despite this, it was feasible to reconstruct reasonably accurate 3D paths with following simple process. Initially, edge detection algorithms are employed to identify clear paths. Since each image provides positional information in two dimensions, binding these paths across the three orthogonal projections was relatively straightforward. Paths that are clear in all three perspectives are averaged, those blurring in one perspective are compensated by the other two, and those blurring in two perspectives are filled by averaging the nearest endpoints to complete the missing dimension. Given the simple setting on environment characterization, network building and training, achieving such results demonstrates the feasibility of using generative models to understand the environment and identify effective paths. The reconstructed path can be overlaid back to 3D scene, shown in Fig. 6.

Although there are many mature and practical workflows for 3D environmental reconstruction based on multimodal environment data, most of these methods are designed for general tasks. Consequently, some details may lead to a significant decrease in the performance of subsequent radio simulations. For instance, many 3D reconstruction methods output models with uneven surfaces that should be flat. While this might not pose a significant issue for tasks like planning that focus on bounding boxes, it can significantly impact radio simulations. The uneven surfaces reconstructed at angles different from the ground truth can cause a doubled deviation in angle of the reflected wave, thus significantly affecting the accuracy of subsequent propagation. Ensuring the accuracy of the reconstructed plane angles will be crucial.

In this article, we conducted a simple proof-of-concept validation for employing generative network on neural radio tracing. However, both in terms of accuracy and generalizability, there is a long distance to go before achieving truly practical neural radio tracing. Another challenge is calculating the channel characteristics after identifying the paths. Identifying the factors that most significantly affect the discrepancy between simulation and reality will require considerable effort in field measurement and the modeling of those factors.

With the aid of holographic communication and digital radio twins, constructing high-level applications to maximize the benefits is a promising avenue. For example, in the context of CAVs and intelligent transportation systems, the features extracted by the on-board computation units from the data collected by sensors can be fed back to BSs to assist in the construction and updating of scenes. Simultaneously, the path and motion planning of CAVs can help the digital radio twin system to predict channels in advance and make resource reservations for critical services.