Neural Surveillance:
Unveiling the Enigma of Latent Dynamics Evolution through Live-Update Visualization

For a full set of representations $\mathbf{R}^{m}_{l}(t)=f_{1:l}(\mathcal{D}_{m})$ trained on the current dataset $\mathcal{D}_{m}$ at epoch $t$, we generate its visualization. The visualization not only needs to preserve the topological structure of $\mathbf{R}^{m}_{l}(t)$, but also the relationship with past representations from different datasets $\mathcal{D}_{m}^{\prime}$ at various epochs $[t-1]$. In particular, we build two graphs of pairwise dependencies: a k nearest neighbor graph $\mathcal{G}_{s}$ between the current representations and a bipartite graph $\mathcal{G}_{b}$ between the current representation and the past context. To simplify and clarify the notation, we refer to the current set of representations as $\mathbf{R}_{\mathcal{T}}$ and the past representations as $\mathbf{R}_{\mathcal{C}}$.

Spatial Relation. Following the approach outlined in [4] for constructing the k nearest neighbor graph, we define $\mathcal{G}_{t}=(\mathbf{R}_{\mathcal{T}},\mathbf{E}_{\mathcal{T}})$ Here, the vertex set $\mathbf{R}_{\mathcal{T}}$ represents the current representation set, and the edge set $\mathbf{E}_{\mathcal{T}}$ is defined as:

The weight function $p(r_{i},r_{j}):\mathbb{R}^{r}\times\mathbb{R}^{r}\rightarrow[0,1]$ denotes the similarity measure between representations in $\mathbf{R}_{\mathcal{T}}$, as specified in Eq. 2.

Temporal Relation. We seek to capture the dynamics of network evolution by a bipartite graph $\mathcal{G}_{b}=(\mathbf{R}_{\mathcal{T}},\mathbf{R}_{\mathcal{C}},\mathbf{E}_{\mathcal{C}})$. Given the complexity, it’s not practical to include representations from all time steps. To address this, we develop a working memory that selectively incorporates partial historical contexts. Specifically, we include representation sets at time step $t^{\prime}$ that occur at intervals of powers of $2$ distance from a current time step $t$. Specifically, the vertex set is defined as:

Note that the representation is not necessarily extracted from a single dataset.

This method ensures that as $t$ increases, the intervals between selected time step $t^{\prime}$ expand. Consequently, this allows for the inclusion of representations from more distant past time steps, therefore capturing long-term memory. Simultaneously, the model includes time steps closer to $t$ for smaller powers of 2, thereby integrating more recent information and addressing short-term memory aspects. The working memory reduces the complexity of the analysis to $\mathcal{O}(\log t)$, effectively balancing the need to preserve critical information with the practicality of computational efficiency.

In defining the edge weight function for the bipartite graph, a key challenge is that representations from different time steps are not directly comparable due to the lack of meaningfulness in their absolute value differences. To address this, we choose cosine similarity as our edge weight function. We employ Cosine similarity for two reasons: 1) The cosine similarity emphasizes directional similarity, which is more meaningful as it captures the change in network evolution rather than just the extent of change; 2) the outcome range is between zero and one, which provides a more normalized and consistent scale. We then arrive at the following formula:

Here, $\mathbf{E}_{\mathcal{C}}$ represents the relationships between the current representation and the past context.

Autoencoders (AEs) are highly effective for dimension reduction [5]. We select AEs as our parametric model for learning the latent structure of our composite graph $\mathcal{G}=\mathcal{G}_{s}\cup\mathcal{G}_{b}$. In this process, we apply the UMAP cost function (see Eq. 4) combined with a reconstruction loss, optimizing both the dimensional reduction and the accuracy of data representation.

Further, without the prior knowledge of the statistical mean and variance of the input representation set $\mathbf{R}_{\mathcal{T}}$, we incorporate a Batch Normalization (BN) layer [6] into our autoencoders (AEs) to enhance the training speed and stability. Formally, with slight abuse of notation, consider the $k$-th layer in the AEs defined as $z_{k}=ReLU(W_{k}x_{k}+b_{k})$, where $x_{k}$ is the input of $k$-th layer, $W_{k}$ is the weight matrix, $b_{k}$ is the bias parameter, and $ReLU(x)=max(x,0)$. We modify this to $z_{k}=ReLU(BN(W_{k}x_{k}+b_{k}))$. However, this introduces a challenge: the representation $\mathbf{R}^{m}_{l}(t)$ within each epoch $t$ experiences shifts in statistical mean and variance. When training the autoencoder with both current and past contexts, these distributional shifts can lead to unstable training outcomes. To overcome this, notice that the Group Normalization (GN) layer performs normalization at the instance level [15]. Therefore we integrate a Group Normalization (GN) layer before the initial BN layer of the encoder and the last BN layer of the decoder.

By coupling GN with BN, our model not only retains the benefits of BN but also overcomes the adverse effects of cross-distribution normalization, leading to more stable and reliable training results.

From Figure 1, it is evident that the visualization quality gradually decreases before sharply declining once the pruning ratio surpasses a certain threshold. Informed by our empirical observations, we propose a “Density-Guided Pruning” approach for our visualization process as Algorithm 1. This decline may result from the loss of crucial topological structures (such as decision boundaries), which becomes difficult to estimate when sample density is too low. Empirical data suggests that high-quality visualization can be effectively accomplished with a more modest number of samples. Consequently, this finding motivates us to consider sample density as an indicator of when to halt the pruning process, demonstrating a distinct correlation.

This algorithm begins by calculating the initial density of the dataset (line 1). Subsequently, a binary search is employed to identify the optimal pruning ratio that aligns with a predefined density threshold (lines 4-11). To enhance the efficiency of this search, we introduce a specified precision level for the pruning ratio, set at $p=0.1$. This precision setting allows the algorithm to converge more rapidly, within four iterations or fewer, to the optimal pruning ratio. Additionally, to mitigate potential variability due to the randomness of pruning, the algorithm calculates the density three times for each iteration and takes the average of these calculations as a basis for decision-making. This approach is particularly effective in data-intensive scenarios.

We run image classification tasks on three datasets, including CIFAR-10 [8], CIFAR-100 [8], and FOOD101 [2]. We selected image datasets with varying image sizes and class numbers to evaluate whether our approach is effective across different scenarios. The details about datasets are in Table 1.

We chose two commonly used architectures as subject models across all datasets: CNN-based models (e.g., ResNet and its variants) and Transformer-based models (e.g., ViT and its variants). To see whether our visualization can work well in different scenarios, we follow two training receipts: 1) training from scratch, 2) fine-tuning from a pre-trained model. For CIFAR10 and CIFAR100 datasets, we train them from scratch using ResNet18/ResNet-34 and ViT with 6 layers and 8 heads respectively for 200 epochs. For FOOD101, we fine tune them on ResNet50 and ViT/B-16 for 20 epochs. More details are in Table 2.

For embedding from high-dimensional space with $d$ dimensions, the visualization model architecture is designed as follows: $(d,\lfloor d/2\rfloor,\lfloor d/4\rfloor\lfloor d/8\rfloor\lfloor d/16\rfloor,2)\&(2,\lfloor d/16\rfloor,\lfloor d/8\rfloor\lfloor d/4\rfloor\lfloor d/2\rfloor,d)$. We set unified hyperparameters for all the training cases. We train it with Adam optimizer with learning rate $1e-2$ and weight decay $1e-5$. The learning scheduler is StepLR with step size $4$ and gamma $0.1$.

We evaluate our SentryCam against multiple baselines, including (1) DVI, a parametric dimension reduction method that employs sequential training, (2) TimeVis, an autoencoder-based method working in post-hoc manner. For the experiments following, we follow the configuration stated in their original paper.

Catastrophic forgetting refers to the phenomenon where a model loses previously acquired knowledge upon learning new information. Figures 8 and 9 display visualizations from a Domain Incremental Learning scenario, utilizing the ER and FROMP strategies respectively, which illustrate the distribution of data from earlier contexts.

These visualizations indicate that both models experience some degree of forgetting as they acquire new tasks, with data from Context 1 exhibiting the most significant loss of detail. Conversely, the data from Context 4 retains a more defined cluster shape, suggesting that more recent contexts are less affected by forgetting. This pattern highlights the models’ varying ability to preserve earlier learned information over successive learning phases.

Task transfer in continual learning assesses a model’s capability to leverage knowledge acquired from previously learned tasks when addressing new, related tasks. Figures 10 and 11 display visualizations of data from later contexts in a Class Incremental Learning setting, specifically starting from Context 1. In the ER strategy, there is noticeable separation among the data points, although significant overlap still exists. This suggests that while there is some retention and differentiation of learned knowledge, there is room for improvement in how distinctly the model can separate new task data from existing contexts. This overlap, however, also implies a blending of features that might foster generalization, indicating a promising direction for future tasks. In contrast, the FROMP strategy exhibits a high degree of overlap among almost all data from later tasks. This extensive overlap could present challenges in future contexts, as it indicates a potential difficulty in distinguishing new task data from previous learning, which may complicate the learning of distinct new tasks.