A Dataset Similarity Evaluation Framework for Wireless Communications and Sensing

The key tool for this transformation is Uniform Manifold Approximation and Projection (UMAP) [16]. Let $D_{i}=\{\mathbf{x}_{j}^{(i)}\}_{j=1}^{M_{i}}$ be a dataset with $M_{i}$ datapoints, where $\mathbf{x}_{j}^{(i)}\in\mathbb{R}^{N}$. UMAP projects this dataset into a lower-dimensional space, represented as $\tilde{D}_{i}=\{\tilde{\mathbf{x}}_{j}^{(i)}=f_{\text{UMAP}}(\mathbf{x}_{j}^{(i)})\}_{j=1}^{M_{i}}$, where the distances better capture local and global structures compared to the raw feature space. UMAP is a dimensionality reduction technique that can be used for general non-linear dimensionality reduction. The algorithm relies on three key assumptions about the data: i) The data is uniformly distributed on a Riemannian manifold, ii) the Riemannian metric is locally constant (or can be approximated as such), iii) the manifold is locally connected. From these assumptions, it is possible to model the manifold with a fuzzy topological structure. UMAP finds a low-dimensional projection of the data that preserves this fuzzy topological structure as closely as possible. The process is provided below.

• •

  Constructing a Fuzzy Simplicial Set: UMAP builds a weighted k-nearest neighbor graph in the high-dimensional space, capturing local neighborhood relationships based on connection probabilities between points.

• •

  Fuzzy Topological Representation: These probabilities are used to construct a topological space, reflecting both local and global structures of the dataset.

• •

  Low-Dimensional Embedding Optimization: UMAP finds a low-dimensional embedding that minimizes the cross-entropy between fuzzy representations in the high-dimensional and low-dimensional spaces.

Preserving both local and global relationships, UMAP provides a better alternative to PCA (which struggles with non-linear structures) and t-SNE [17] (which distorts global structures). This makes it ideal for computing distances that reflect how models trained on one dataset perform on another.

Euclidean in UMAP Spaces: In the UMAP-encoded latent space, various forms of Euclidean distances are utilized to quantify dataset separation, providing computational simplicity and interpretability:

Pairwise Euclidean Distance: The distance between all pairs of points across two datasets $D_{1}$ and $D_{2}$, where $M_{1}$ and $M_{2}$ are the number of data points in the datasets, is given by

$$d_{\text{pairwise}}=\frac{1}{M_{1}M_{2}}\sum_{j=1}^{M_{1}}\sum_{k=1}^{M_{2}}\left\|\tilde{\mathbf{x}}_{j}^{(1)}-\tilde{\mathbf{x}}_{k}^{(2)}\right\|_{2}.$$

Here, $\tilde{\mathbf{x}}_{j}^{(1)}$ and $\tilde{\mathbf{x}}_{k}^{(2)}$ represent the encoded points from datasets $D_{1}$ and $D_{2}$, respectively.

Cluster-Based Euclidean Distance: After clustering each dataset into $K_{1}$ and $K_{2}$ clusters, the centroid-based Euclidean distance is computed as

$$d_{\text{cluster}}=\frac{1}{K_{1}K_{2}}\sum_{l=1}^{K_{1}}\sum_{m=1}^{K_{2}}\left\|\mathbf{c}_{l}^{(1)}-\mathbf{c}_{m}^{(2)}\right\|_{2},$$

where $\mathbf{c}_{l}^{(1)}$ and $\mathbf{c}_{m}^{(2)}$ are the centroids of clusters $l$ and $m$ in datasets $D_{1}$ and $D_{2}$, respectively.

Average Euclidean Distance: This distance simplifies to the Euclidean distance between the mean vectors $\bar{\mathbf{x}}^{(1)}$ and $\bar{\mathbf{x}}^{(2)}$ of the two datasets

$$d_{\text{average}}=\left\|\bar{\mathbf{x}}^{(1)}-\bar{\mathbf{x}}^{(2)}\right\|_{2},$$

where $\bar{\mathbf{x}}^{(1)}=\frac{1}{M_{1}}\sum_{j=1}^{M_{1}}\tilde{\mathbf{x}}_{j}^{(1)}$ and similarly for $\bar{\mathbf{x}}^{(2)}$.

The advantage of Euclidean distances in UMAP spaces lies in their computational efficiency and ability to clearly reflect local dataset structure after dimensionality reduction, which is crucial for fast and intuitive dataset similarity evaluation.

Wasserstein in UMAP Spaces: The Wasserstein distance [18, 19, 20], also known as the Earth Mover’s distance, is a more powerful metric for measuring differences between two distributions. It calculates the minimal effort needed to transport the mass of one distribution to match the other across all dimensions of the latent space. For each dimension $n$, the one-dimensional Wasserstein distance between the cumulative distribution functions (CDFs) $F_{n}^{(1)}$ and $F_{n}^{(2)}$ of datasets $D_{1}$ and $D_{2}$ is

$$W_{n}=\int_{0}^{1}\left|F_{n}^{(1)^{-1}}(t)-F_{n}^{(2)^{-1}}(t)\right|dt,$$

where $F_{n}^{(i)^{-1}}(t)$ is the inverse CDF (quantile function) for the $n$-th dimension of dataset $D_{i}$.

The overall Wasserstein distance across all dimensions $d$ of the latent space is then the average of these one-dimensional Wasserstein distances as follows

$$W=\frac{1}{d}\sum_{n=1}^{d}W_{n}.$$

Wasserstein distance is especially advantageous for capturing the global structure of datasets, providing superior sensitivity to both local and global distributional differences. This makes it highly effective in UMAP spaces, where preserving both local and global features is critical for accurate dataset similarity evaluation.

UMAP Considerations: UMAP requires careful tuning of parameters, such as the number of neighbors and minimum distance in the latent space. A balance between local and global structures must be achieved for effective embedding. For wireless datasets, correlation distance has proven effective for UMAP, but Euclidean distance can be used when scalability is prioritized over performance. Leveraging UMAP for distance computation offers improved accuracy in assessing dataset similarities, providing a robust foundation for dataset distancing tasks.

CSI compression involves transforming a high-dimensional wireless channel matrix, $\mathbf{H}\in\mathbb{C}^{N_{BS}\times N_{sub}}$, into a compact, low-dimensional representation. Here, we consider $N_{BS}$ basestation antennas and $N_{sub}$ subcarriers in the channel matrix. After applying a Fourier transform and truncating the last 16 delay taps, the matrix is reduced to $32\times 16$. The goal is to compress this matrix to $N_{enc}=32$ dimensions, achieving a $64$-fold reduction. The compression performance is measured using the normalized mean squared error (NMSE) between the original matrix $\mathbf{H}$ and the reconstruction $\hat{\mathbf{H}}$ as follows

$$\text{NMSE}_{\text{dB}}(\mathbf{H},\hat{\mathbf{H}})=10\log_{10}\frac{\|\mathbf{H}-\hat{\mathbf{H}}\|^{2}_{F}}{\|\mathbf{H}\|^{2}_{F}}.$$

The autoencoder (AE) used for CSI compression processes the input $32\times 16$ matrix, split into real and imaginary parts, through convolutional layers to reduce it to a $32$-dimensional latent space. The decoder reconstructs the high-dimensional matrix from the latent space. The model is trained using MSE loss and is configured in two ways:

• •

  AEs are trained separately on each area dataset. Each model achieves an NMSE below $-20\text{dB}$ when tested on the trained area and higher NMSE on untrained areas. These 20 models are used to access performance drops that should be correlated with our distance metrics.

• •

  A single AE with five refinement nets (instead of three) is trained on data from all areas, resulting in an average NMSE of $-20\text{dB}$ across areas. This model latent space is used for computing dataset distances later on when analyzing the impact of dimensionality reduction methods.

The dataset used in this work is a raytraced ASU campus dataset generated with the DeepMIMO framework, containing around $90K$ users across an area of $410\times 320$ meters with 1-meter resolution. The raytracing simulation captures various propagation effects, including LoS, reflections, diffuse scattering, and diffraction, providing a highly realistic environment. Figure 3 compares the real geographic view of the ASU campus with the digitally rendered version, illustrating the dataset’s suitability for evaluating distance metrics in wireless channel compression tasks.

Table I presents the correlations between different distance metrics and model performance for the CSI compression task in the raw input space. We evaluate three categories of distance metrics: geometric, statistical, and subspace-based. Key findings include:

• •

  Statistical distances outperform geometric ones. Wasserstein ($0.52$ correlation) and Energy ($0.55$ correlation) distances perform best, because they capture distributional differences, unlike point-to-point metrics that struggle with high-dimensional data.

• •

  Geometric distances show lower correlations. Averaged and pairwise Euclidean distances achieve roughly $0.35$ correlation, due to their limitations for high-dimensional wireless data.

• •

  Subspace distances performed poorly Grassmann, Chordal, and Asimov metrics perform the worst and even yield negative correlations. This can be associated with the similarity between projected subspaces, resulting in practically constant principal angles and, thus, distances.

• •

  Computation time matters. While Wasserstein and Energy perform well, they are computationally expensive (562s and 735s, respectively). Simpler methods like centroid Euclidean (3s) are faster but offer lower correlations.

These results highlight that while statistical distances are more effective, their computational cost motivates the need for dimensionality reduction to make distance computations more practical.

Dimensionality reduction helps mitigate the high computational costs of raw space distance computation by removing noise and redundancies. Fig. 5 compares several techniques, including PCA, t-SNE, and UMAP.

UMAP was found to be the most effective among dimensionality reduction techniques, striking a balance between preserving local and global structures. PCA, being linear, fails to differentiate between similar datasets, while t-SNE distorts global relationships by focusing on local clusters. UMAP retains essential geometric features, resulting in a latent space where distances better reflect dataset similarities.

Table  II summarizes the performance of various distance metrics in different latent spaces, using PCA, t-SNE, UMAP, and autoencoder (AE).

AE as an upper bound: The AE-based latent space achieves the highest correlation ($0.94$) between distances and model performance. This is unsurprising, as the AE architecture used for dimensionality reduction is similar to the model used for performance evaluation. However, the downside of this approach is its impracticality. Training an AE for all datasets is computationally expensive and time-consuming, making it an infeasible solution for real-world applications.

UMAP as a practical alternative: UMAP provides a close approximation to the AE’s performance, with correlations of around 0.85 for both Euclidean and Wasserstein distances. UMAP’s ability to balance local and global structures makes it a highly effective dimensionality reduction technique. Importantly, UMAP drastically reduces the computational complexity of distance computations, making it a practical choice for large-scale applications.

Euclidean and Wasserstein metrics perform best: In both the UMAP and AE spaces, Euclidean-based distances and Wasserstein distance achieve the highest correlations. This suggests that these metrics are well-suited to the latent spaces generated by UMAP and AEs, capturing meaningful dataset similarities that correlate with model performance.

Dimensionality reduction reduces computational costs: By projecting the data into lower-dimensional latent spaces, we reduce the computational burden of distance computation. For example, computing Wasserstein distance in the raw space takes 562s, but this time is significantly reduced when using UMAP. This makes UMAP a practical choice for improving both performance and computational efficiency.

The choice of dimensionality reduction method plays a crucial role in improving both the accuracy and efficiency of distance computations. While AE-based latent spaces provide the highest correlation with model performance, UMAP emerges as a more practical alternative, offering strong correlations with much lower computational overhead. UMAP, combined with simple Euclidean or Wasserstein distances, achieves correlations of around 0.85 with model performance, providing a close approximation to the upper bound set by the AE. This makes UMAP an attractive choice for real-world applications, where computational efficiency and scalability are critical considerations.