ChronoGAN: Supervised and Embedded Generative Adversarial Networks for Time Series Generation

In a joint training scheme involving a GAN network and an autoencoder, relying solely on reconstruction loss for the autoencoder results in noisy outputs, where the autoencoder’s output fails to fully retain the input’s characteristics [21]. Additionally, adversarial training within an embedding space leads to the generation of noisy data after decoding the generator’s output. The issue arises when the encoder’s output ($H^{AE}$) is regarded as real data and the generator’s output ($H^{G}$) as synthetic during the adversarial training process. This practice reduces the discriminator’s ability to accurately differentiate between the attributes of real and synthetic data. A significant limitation is that the discriminator does not account for the error rate and data loss inherent in the autoencoder’s performance. This oversight may compromise the efficacy of the discriminator, resulting in suboptimal performance in distinguishing between real and generated data attributes. Consequently, this leads to less precise feedback being provided to the generator network, potentially affecting the overall quality of the synthetic data. To address this, as shown in Fig. 1, discriminating in the feature space allows for defining real data as the dataset ($X$) and fake data as the decoding of the generator’s output ($X^{G}$). This facilitates more accurate training for the discriminator, thus yielding improved feedback for the generator. Additionally, discrimination in the feature space provides valuable adversarial feedback to the autoencoder, enhancing its reconstruction capabilities in conjunction with conventional reconstruction loss. In the context of time series data, the feature space denotes the original dimensions, such as individual time points and their observed values. The latent or embedding space, achieved through an encoding process, represents the data in a lower-dimensional form, capturing its essential patterns and structures in a more compact and informative manner [22].

Through a joint learning scheme, the autoencoder is initially trained using a combination of reconstruction loss and binary feedback from the discriminator, where real data is the dataset ($X$) and fake data is its reconstruction ($X^{AE}$). This approach enhances the autoencoder’s precision in reconstructing outputs. In the subsequent phase, only the supervisor network is trained. The supervisor utilizes real data embeddings from the previous two time steps $h_{1:t-2}$ generated by the embedding network to create the subsequent latent vector $h_{t}$. Finally, all five networks are trained jointly. During this final phase, the same discriminator distinguishes between real data, denoted as the dataset ($X$), and the dataset reconstructions ($X^{AE}$), where the fake data comprises the generator’s decoded outputs ($X^{G}$) and the supervisor’s decoded outputs ($\tilde{X}$). The generator undergoes training through this adversarial feedback $\mathcal{L}_{U}$, in addition to other feedback mechanisms including $\mathcal{L}_{S}$, $\mathcal{L}_{V}$, and $\mathcal{L}_{TS}$. This phase involves a shift in the characterization of fake and real data compared to the initial phase.

Based on the feedback from the discriminator, we introduce a new loss function for the autoencoder ($\mathcal{L}_{AE}$), which comprises both reconstruction loss ($\mathcal{L}_{R}$) and adversarial loss ($\mathcal{L}_{U}$). The proportion of reconstruction loss to adversarial loss decreases in the third phase of training compared to the first phase, where the primary purpose of the discriminator is to provide feedback for the generator rather than the autoencoder.

Where $t$ denotes an individual time step, and $T$ represents the total number of time steps within the series. In addition, $\mathbf{x}_{t}$ represents the real data at timestamp $t$, and $\mathbf{x}^{AE}_{t}$ denotes the output of the autoencoder corresponding to the real data $\mathbf{x}_{t}$ at the same timestamp.

Here, $p$ indicates the probability distribution of real data, and $\tilde{p}$ represents the probability distribution of synthetic data. Moreover, the discriminator $d$ generates the output $\tilde{y}$ when evaluating the autoencoder’s output $X^{AE}$ and produces the output $y$ when assessing the real samples $X$

The sole reliance on the discriminator’s binary adversarial feedback might not sufficiently drive the generator to capture the data’s stepwise conditional distributions. To address this, ChronoGAN introduces an additional component, the supervisor, along with a novel loss mechanism denoted by $\mathcal{L}_{S}$. ChronoGAN employs a closed-loop training mode, where the supervisor utilizes actual data embeddings from the previous two time steps $h_{1:t-2}$ produced by the embedding network to generate the subsequent latent vector $h_{t}$. This looped training involves the generator’s loss $\mathcal{L}_{G}$, which encompasses the adversarial loss $\mathcal{L}_{U}$, the stepwise transition loss $\mathcal{L}_{S}$, the distribution loss $\mathcal{L}_{V}$, and our innovative time series loss $\mathcal{L}_{TS}$. This structure ensures the generation of realistic sequences with accurate temporal transitions. The distribution loss $\mathcal{L}_{V}$ leverages the mean absolute error (MAE) of the mean and variance between the real data $X$ and the generated data $\tilde{X}$. This approach effectively assists the generator in learning the real data distribution, enabling it to produce data across the entire distribution, which also serves as a key metric for evaluating GAN techniques.

Where $\mathcal{L}_{Mean}$ is the MAE of the mean between a batch of real and generated samples, and $\mathcal{L}_{Variance}$ is the MAE of the variance between the same batch of real and generated data.

Where each sample is labeled by $n\in\{1,\ldots,N\}$ and the batch is represented as $\mathcal{B}=\{\mathbf{x}_{n,1:T_{n}}\}_{n=1}^{N}$.

Where $\overline{\mathbf{x}}$ indicates the mean of $\mathbf{x}$, and $\overline{\mathbf{\tilde{x}}}$ represents the mean of $\tilde{\mathbf{x}}$ for a batch of data.

Where $s$ is the supervisor network, $h^{G}_{t}$ is the output of the generator at timestamp $t$, and $h^{G}_{t-2}$ is the output of the generator at timestamp $t-2$. This technique is more efficient than predicting timestamp $t$ using timestamp $t-1$.

In the third phase of training, referred to as joint training, $\tilde{y}$ represents the output of the discriminator $d$ for synthetic samples $X^{G}$ and $\tilde{X}$, while $y$ denotes the output of $d$ for real samples $X$ and $X^{AE}$.

Furthermore, we introduce a novel loss function for the generator called the time series loss, $\mathcal{L}_{TS}$, which not only facilitates convergence but also enhances the quality of the generated data. This loss function is defined as the mean squared error (MSE) of the mean and standard deviation (std) of four key time series characteristics, including slope, skewness, weighted average, and median, between real and synthetic data. The aim is to boost the generator’s convergence and its ability to learn the real data characteristics and distribution, as relying solely on the adversarial loss is insufficient for learning the characteristics of real time series data. The time series loss $\mathcal{L}_{TS}$ is a novel contribution, comprising the slope loss ($\mathcal{L}_{Slope}$), weighted average loss ($\mathcal{L}_{WeightedAvg}$), skewness loss ($\mathcal{L}_{Skewness}$), and median loss ($\mathcal{L}_{Median}$).

The slope loss $\mathcal{L}_{Slope}$ includes the MSE of the mean ($\mathcal{L}_{S_{mean}}$) and the MSE of the std ($\mathcal{L}_{S_{std}}$) between the slopes of real and generated samples.

The slope is calculated using the provided formula,

In these equations, $S$ is the slope of real samples, and $\tilde{S}$ is the slope of generated samples.

Other components of $\mathcal{L}_{TS}$, such as skewness, weighted average, and median, are calculated similarly to (10), (12), and (III-B). The only difference is that instead of using the formula for slope, the formulas for skewness (skew), weighted average (wAvg), and median are applied.

Where $\sigma_{x}$ represents the std of $x$, and $w_{t}$ denotes the weight assigned to the value $x_{t}$ at timestamp $t$.

Leveraging the strengths of different neural network architectures by combining them has long been a powerful and effective approach. In auditory attention detection (AAD), combining GRU and CNN architectures has been particularly effective. CNNs, while good at extracting spatial features from EEG data, struggle to capture long-term dependencies. To address this, the AAD-GCQL model [1] integrates GRU with CNN to capture both spatial and temporal dynamics in EEG signals, enhancing the detection of auditory attention.

The GRU used in this combination belongs to a broader family of recurrent neural networks (RNNs), which are tailored for sequence modeling tasks. Among these, LSTM and GRU are the two most prominent architectures, frequently applied in domains such as natural language processing [23] and time series forecasting. LSTMs are equipped with memory cells and three distinct gates (input, output, and forget), which help manage the flow of information and address the vanishing gradient problem seen in traditional RNNs [24]. This architecture makes LSTMs particularly well-suited for longer sequence data, where maintaining information over extended intervals is critical. On the other hand, GRUs simplify the structure by merging the input and forget gates into a single update gate, complemented by a reset gate that determines the extent of past information retention [25]. GRUs tend to be more efficient and quicker to train, making them ideal for tasks with shorter sequences or when computational resources are limited. The decision between using LSTM and GRU often hinges on the specific sequence length and complexity of the task, with LSTMs generally preferred for longer sequences and GRUs for shorter ones [26].

A time series generation framework should be capable of handling both short and long sequences and, more importantly, be accurate on both. The exclusive use of either LSTM or GRU as the network architecture can lead to weaknesses in handling either long or short sequences. As shown in Fig. 2, by implementing both network architectures and merging the results via a multilayer perceptron, the network becomes more generalized, making it more powerful in learning both long and short sequences. We employ multiple layers of GRU and LSTM separately to produce output, and then merge them using a multilayer perceptron network to obtain the final output. We utilize the same architecture and number of layers for all five networks within the ChronoGAN framework.

Another prevalent issue with GANs is stability. To enhance the stability of the network, we employ an early generation algorithm since the optimal results may be achieved after a random, rather than a specific, number of iterations. Accordingly, as per Algorithm 1, after half the number of epochs, we generate synthetic data and calculate the discriminative score and predictive score between real and synthetic data at intervals of every 500 epochs. Additionally, we compute the MSE of the mean and MSE of the std of real and synthetic data to verify whether the synthetic data matches the distribution of the real data. By integrating the results of the discriminative score, predictive score, and MSE of the mean and std, we determine whether to save the current model and generated data. Upon the completion of training, we ensure that the framework has produced the optimal results, consistently delivering reliable and precise outcomes after each training session. It is crucial to determine the appropriate weights for these metrics in order to integrate them and compare them with the previously saved model. The proportion of the discriminative score, predictive score, and MSE of the mean and std can vary depending on the characteristics of the dataset. Therefore, it is inappropriate to establish fixed hyperparameters to combine these three metrics. To address this issue, we initially calculate the hyperparameters $p1$ and $p2$ during the first assessment of these metrics. Once established, these hyperparameters are consistently applied in all subsequent epochs.

We evaluate ChronoGAN’s effectiveness on time-series datasets with varying attributes such as periodicity, discreteness, noise levels, length, and feature correlation over time. We choose the datasets based on different combinations of these characteristics:

1. 1.

   Stocks: Stock price sequences are continuous but aperiodic and features are correlated. We use daily historical data from Google stocks spanning 2004 to 2019, which includes features such as volume, high, low, opening, closing, and adjusted closing prices.

2. 2.

   Sines: We generate multivariate sinusoidal sequences with varying frequencies $\eta$ and phases $\theta$, providing continuous, periodic, and multivariate data with each feature being independent.

3. 3.

   ECG: The ECG5000 dataset from Physionet, which covers a 20-hour long ECG recording with 140 timestamps, is a univariate time series that is continuous and periodic. The data is classified as a long time series.

4. 4.

   SWAN-SF: The Space Weather Analytics for Solar Flares (SWAN-SF) [27] dataset consists of multivariate time series of photospheric magnetic field parameters for solar flare prediction tasks [28]. The SWAN-SF dataset is recognized as challenging due to its complex temporal dynamics and the numerous data preprocessing issues it presents. In [29], the authors thoroughly addressed these challenges by implementing an innovative preprocessing pipeline [30]. This effort resulted in the creation of an enhanced version of the SWAN-SF dataset [31], which was subsequently utilized in our evaluation in place of the original, unprocessed dataset.

We conduct a comparison between ChronoGAN, TimeGAN [16], Teacher Forcing (T-Forcing) [19], Professor Forcing (P-Forcing) [18] and Standard GAN [13], which represent the five best-performing techniques in various fields of time series generation, including GAN-based and Autoregressive approaches. To ensure unbiased results, we maintain identical hyperparameters across all five models. To evaluate the quality of the generated data, we focus on three key criteria:

1. 1.

   Visualization: We utilize t-SNE [32] and PCA [33] analyses on both the original and synthetic datasets. This approach aids in qualitatively assessing how closely the distribution of the generated samples matches that of the original in a two-dimensional space.

2. 2.

   Discriminative Score: For a quantitative measure of similarity, each sequence from the original dataset is labeled as ‘real‘, while each from the generated set is labeled as ‘synthetic‘. An LSTM classifier is then trained to differentiate these two categories in a standard supervised learning task. The classification error on a reserved test set provides a quantitative measure of this score. We then subtract the result from 0.5, making the optimal result 0 instead of 0.5 for easier comparison.

3. 3.

   Predictive Score: To evaluate the quality of the generated data in capturing step-wise conditional distributions, we utilize the synthetic dataset to train an LSTM for sequence prediction. This involves forecasting the next-step temporal vectors for each input sequence. The model’s accuracy is subsequently tested on the original dataset, with performance assessed using the MAE.

For each discriminative or predictive score experiment, we replicated the experiments eight times to avoid incidental results. We present the mean and std of each experiment in Tables I and II.

Based on the results presented in Tables I and II, the ChronoGAN framework consistently outperforms state-of-the-art models, including TimeGAN, Teacher Forcing, Professor Forcing, and Standard GAN. In terms of the discriminative score, ChronoGAN achieves an average reduction of approximately 27.60% across the four datasets compared to TimeGAN. This substantial improvement indicates that ChronoGAN generates more realistic temporal data than other techniques. Furthermore, this improvement in the discriminative score can be attributed to the early generation algorithm, which enhances stability and ensures the best data is preserved during training. The improvement is also evident across all four datasets, each with different lengths, demonstrating the effectiveness of the GRU-LSTM layers within our framework. Additionally, according to discriminative score evaluations, ChronoGAN and TimeGAN emerge as superior compared to Teacher Forcing and Standard GAN. This underscores the importance of developing GAN-based techniques specifically tailored for time series data.

In terms of predictive score, ChronoGAN reduces the error by approximately 10.82% across the four datasets compared to TimeGAN. This underscores the effectiveness of our novel time series-based ($\mathcal{L}_{TS}$) and supervised ($\mathcal{L}_{S}$) loss functions, which significantly improve the generator’s ability to capture the temporal dynamics of the data more accurately. As demonstrated in Figs. 3 and 4, we present several examples of synthetic samples generated by ChronoGAN for both the Sines and ECG datasets. These examples highlight ChronoGAN’s ability to effectively learn the temporal distributions of the real data and generate high-quality synthetic data that accurately reflect those patterns.

Based on Figs. 5 and 6, ChronoGAN demonstrates a superior ability to learn the probability distribution of real datasets more efficiently than all other baseline techniques. This is crucial, as a GAN-based model must generate data that accurately covers the entire distribution of the real dataset. The PCA and t-SNE results for the Stocks dataset show highly accurate outcomes. This achievement is primarily due to the $\mathcal{L}_{V}$ loss, which enables the network to effectively capture the mean and variance of each batch of real data.