CausalStock: Deep End-to-end Causal Discovery
for News-driven Stock Movement Prediction

In CausalStock, we integrate the model inputs with causal relations into FCM for prediction. In this section, we introduce the fundamental concepts of FCM and the temporal causal graph.

In this paper, we focus on tackling the news-driven multi-stock movement prediction task. For the target trading day $T$, we denote the model inputs as the past $L$ time lag information of $D$ stocks as $\bm{X}_{<T}=\{X_{t}^{i}\}_{t=T-L:T-1}^{i=1:D}=[\bm{C}_{<T},\bm{P}_{<T}]=\{[C_{t}^{i},P_{t}^{i}]\}_{t=T-L:T-1}^{i=1:D}$, where $C_{t}^{i}$ and $P_{t}^{i}$ represent the news corpora representation and the historical price features representation of $i$-th stock at time step $t$ respectively. The objective is to predict the movement of adjusted close prices $\bm{y}_{T}=\{y_{T}^{i}\}_{i=1}^{D}\in\mathbb{R}^{D\times 1}$ on $T$-th trading day of all stocks simultaneously, where $y_{T}^{i}\in\{0,1\}$ representing the $i$-th stock price will fall or rise at trading day $T$, i.e., stock movement. In a theoretical way, this task could be trained by maximizing the log-likelihood of conditional probability distribution $p\left(\bm{y}_{T}\mid\bm{X}_{<T}\right)$, so that the most likely $\bm{y}_{T}$ are generated.

The conditional probability distribution could be further factorized as follows:

The overall process is taken as two joint training parts: temporal causal graph discovery $p\left(\bm{G}\mid\bm{X}_{<T}\right)$ and the prediction process given the causal relations $p\left(\bm{y}_{T}\mid\bm{X}_{<T},\bm{G}\right)$. The probabilistic graphic representation of this modeling process is shown in Figure 1. In CausalStock, we develop a lag-dependent causal discovery module, according to which we could take another step by modeling $p\left(\bm{G}\mid\bm{X}_{<T}\right)$ as a lag-dependent format:

For the prediction part $p\left(\bm{y}_{T}\mid\bm{X}_{<T},\bm{G}\right)$, we design an FCM as shown in Equation 9 to predict the future movement based on the past information $\bm{X}_{<T}$ and the discovered temporal causal graph $\bm{G}$.

In a nutshell, CausalStock comprises three primary components as shown in Figure 2:

1. 1.

   Market Information Encoder (MIE) encodes the news text and price features. In this part, an LLM-based Denoised News Encoder is proposed;

2. 2.

   Lag-dependent Temporal Causal Discovery (Lag-dependent TCD) module leverages variational inference to mine the causal relationship based on the given market information of stocks, i.e., modeling $p\left(\bm{G}\mid\bm{X}_{<T}\right)$;

3. 3.

   Functional Causal Model (FCM) generates the prediction of future price movements according to the discovered causal graph, i.e., modeling $p\left(\bm{y}_{T}\mid\bm{X}_{<T},\bm{G}\right)$.

Market Information Encoder (MIE) takes news corpora and numerical stock price features as inputs, and outputs the historical market information representations $\bm{X}_{<T}=[\bm{C}_{<T},\bm{P}_{<T}]=\{[C_{t}^{i},P_{t}^{i}]\}_{t=T-L:T-1}^{i=1:D}=\{X_{t}^{i}\}_{t=T-L:T-1}^{i=1:D}$ for $D$ stocks with time lag $L$. For $i$-th stock, each time step representation $X_{t}^{i}$ is the combination of the text representation $C_{t}^{i}$ generated by the news encoder and the historical price features representation $P^{i}_{t}$ generated by the price encoder.

In this section, we propose Lag-dependent Temporal Causal Discovery module. Inspired by [14], our model takes a Bayesian view for modeling the distribution of temporal causal graph, which aims to learn the posterior distribution $p\left(\bm{G}\mid\bm{X}_{<T}\right)$. Unfortunately, the exact graph posterior is intractable because of the large combination space of $\bm{G}$. Here we adopt the variational inference [3] to get the approximator $q_{\phi}\left(\bm{G}\right)$, where $\phi$ indicates the parameter set of variational inference.

In this section, we design an FCM to model $p_{\theta}\left(\bm{y}_{T}\mid\bm{X}_{<T},\bm{G}\right)$, where $\theta$ denotes the parameter set of FCM. We focus on additive noise FCM [18] to generate $\bm{y}_{T}=\{\bm{y}^{i}_{T}\}_{i=1}^{D}\in\mathbb{R}^{D\times 1}$:

where $z^{i}_{t}$ represents mutually and serially independent dynamical noise, and $f_{i}:\mathbb{R}^{D\times L}\xrightarrow{}\mathbb{R}^{1}$ are general differentiable non-linear function that satisfies the relations specified by the temporal causal graph $\bm{G}$ strictly, namely, if $X_{t}^{j}\notin\mathbf{Pa}_{G}^{i}(<T)$, then $\partial f_{i}/\partial X_{t}^{j}=0$.

We design a novel FCM to aggregate market information including news and prices based on the discovered causal graph $\bm{G}$ and causal weight graph $\bm{\hat{G}}$:

where $\zeta_{i}$, $\ell$ and $\psi$ are all neural networks. $\ell$ and $\psi$ are shared weights across nodes and lags for efficient computation. $[\cdot,\cdot]$ denotes the concatenate operation. We apply the logistic Sigmoid function to output the movement probability of $\bm{y}_{T}$ and use it directly as the output of CausalStock.

For the exogenous noise $\bm{z}_{T}^{i}$ modeling, we adopt Gaussian distribution, i.e., $z_{T}^{i}\sim\mathcal{N}\left(0,\left(\sigma^{i}\right)^{2}\right)$, where per-variable variances $\left(\sigma^{i}\right)^{2},i\in\left[1,D\right]$ are trainable parameters to represent the uncertainty part. According to Change of variables formula [18], the conditional distribution $p_{\theta}\left(y_{T}^{i}\mid\mathbf{Pa}^{i}_{\bm{G}}\left(<t\right)\right)$ could be represented as:

where $p_{z_{i}}$ is the aforementioned Gaussian distribution for stock $i$. $\left|\frac{\partial F_{i}}{\partial z_{T}^{i}}\right|$ indicates the absolute value of the Jacobian-determinant for $F_{i}$, $\left|\frac{\partial F_{i}}{\partial z_{T}^{i}}\right|^{-1}=1$ is derived according to Equation 9. Now the log likelihood $\log p_{\theta}\left(\bm{y}_{T}\mid\bm{X}_{<T},\bm{G}\right)$ could be further represented as:

We train our model by maximizing the conditional log-likelihood $\log p_{\theta}\left(\bm{y}_{T}\mid\bm{X}_{<T}\right)$. The variational evidence lower bound (ELBO) of the model objective is derived as follows:

Here, $p\left(\bm{G}\right)$ represents the prior of causal graph, and $H\left(q_{\phi}\left(\bm{G}\right)\right)$ is the entropy of the posterior approximator. $\log p_{\theta}\left(\bm{y}_{T}\mid\bm{X}_{<T},\bm{G}\right)=\log p_{z}\left(\bm{z}_{T}\right)$ is the log-likelihood of the target distribution, in which $\bm{z}_{T}$ is calculated by Equation 9 at training stage.

Besides, we further adopt the binary cross entropy loss as another objective $\textit{BCE}\left(\bm{g}_{T},\bm{y}_{T}\right)$ to improve the learning performance, where $\bm{g}_{T}$ is the ground truth movement at target trading day $T$. Overall, the final training loss $\mathcal{L}$ is as follows,

where $\lambda$ is the scalar weight to balance loss terms. We note that the required assumptions and the theoretical guarantees are summarized in Appendix B.

Except for the news-driven multi-stock movement prediction task, our model could also handle the multi-stock movement prediction task without news by removing the Denoised News Encoder. Thus, we do the experiments for both two tasks.

Dataset (Appendix C.1): We train and evaluate our model and baselines on six datasets: ACL18 [42], CMIN-US [23], CMIN-CN [23], KDD17 [45], NI225 [44], and FTSE100 [44]. The first three of them including both historical prices and text data are used for news-driven multi-stock movement prediction task evaluation, while the last three are for multi-stock movement prediction task evaluation without news data. Evaluation metrics (Appendix  C.2): We evaluate the prediction performance of models by Accuracy (ACC) and Matthews Correlation Coefficients (MCC). Baselines (Appendix  C.3): HAN  [15], Stocknet  [42], PEN  [21], CMIN  [23] for news-driven multi-stock movement prediction task. LSTM  [25], ALSTM  [28], Adv-ALSTM  [8], DTML  [44] for multi-stock movement prediction task. Parameter setup (Appendix C.4): Our model is implemented with Pytorch on 4 NVIDIA Tesla V100 and optimized by Adam [20]. The parameter sensitivity study can be found in Appendix C.4.

As shown in the top half of Table 1, CausalStock outperforms all baselines on ACC as well as MCC across three datasets demonstrating robustness performance for news-driven multi-stock movement prediction task. For the multi-stock movement prediction task, the results are reported in the bottom half of Table 1. As can be seen, CausalStock exceeds all baselines across three datasets with stable performance. Overall, the results demonstrate that the proposed CausalStock can indeed improve the performance for two stock movement prediction tasks, showing the strong capabilities in handling financial texts and discovering causal relations among stocks.

For the ablation study, we conduct several model variants on ACL18, CMIN-CN and CMIN-US to explore the contributions of different settings in CausalStock. For the main framework, we have the following five variants. CausalStock w/o TCD: removing the causal discovery module from CausalStock; CausalStock w/o News: removing the news encoder from CausalStock and just taking prices data as input; CausalStock w/o link non-existence modeling: only model the causal link existence likelihood and leverage Sigmoid function to obtain the link existence probability; CausalStock w/o Lag-dependent TCD: replacing the Lag-dependent Temporal Causal Discovery module with the Lag-independent Temporal Causal Discovery module; CausalStock with Variable-dependent TCD: we add a variable-dependent causal mechanism that explicitly captures the dependencies among different stock edges. Specifically, each edge’s probability is conditioned on the states of all other edges at the same time step, and the conditional function is the same as the function in the lag-dependent mechanism (Equation 6). Furthermore, we explore the performance of six different Traditional News Encoders by replacing the denoised news encoder, which outputs the news embeddings as representations. CausalStock with Glove + Bi-GRU: leveraging the Glove word embedding and the Bi-GRU as news encoder [23]; CausalStock with Bert: leveraging the pre-trained Bert (Bert-base-multilingual-cased [6]) as news encoder; CausalStock with Roberta: leveraging the pre-trained Roberta (Roberta-base [22]) as news encoder; CausalStock with FinBert: leveraging the pre-trained FinBert [1] as news encoder; CausalStock with FinGPT: leveraging the pre-trained FinGPT (FinGPT-v3.3 [43]) as news encoder; CausalStock with Llama: leveraging the pre-trained Llama ( Llama-7b-chat-hf [39]) as news encoder to output news embeddings. Moreover, we explore the performance of three different LLMs for the denoised news encoder. CausalStock with FinGPT: leveraging a financial LLM FinGPT (FinGPT-v3.3 [43]) as denoised news encoder; CausalStock with Llama: leveraging Llama (Llama-7b-chat-hf [39]) as denoised news encoder. The ablation study results are summarized in Table 2.

We have the following observations: (1) CausalStock with news encoders all perform better than CausalStock without news, suggesting news data is particularly helpful for stock movement prediction. (2) Compared to CausalStock w/o Lag-independent TCD, CausalStock with Lag-dependent TCD has a better performance, demonstrating the value of the lag-dependent mechanism. (3) By comparing the CausalStock and CausalStock with Variable-dependent TCD, the results show that incorporating a variable-dependent causal mechanism has the potential to enhance model performance. However, the improvements are not uniform and vary depending on the dataset, which emphasizes that further validation is needed. While the above results show a promising performance of the variable-dependent causal mechanism, it significantly increases the computational complexity (from $O(L\times D^{2})$ to $O(L\times D^{4})$), making it challenging to apply the model to markets with large numbers of stocks. (3) By using FinGPT and Llama as the news encoder and denoised news encoder respectively, we can observe that denoised news encoders have a relatively higher ACC and MCC than their as the traditional news encoders, suggesting the value of denoised news encoders. Overall, the ablation studies show that every component contributes to CausalStock.

Here, we present many cases detailing the interpretability of CausalStock from two perspectives: the news representation from the Denoised News Encoder, and the causal graph discovered by the Lag-dependent TCD module.

Firstly, regarding the Denoised News Encoder module, three cases are selected as shown in Figure 3(c). A piece of news about APPL suggests a potential delay in its 5G iPhone launch, with Denoised News Encoder giving it a negative sentiment score of $-0.7$ and an impact score of $9$. Similarly, a news about TSLA hints at surpassing a significant delivery milestone, receiving a positive sentiment score of $0.7$. In contrast, a news piece showing no discernible connection to GOOG is scored with negligible impact. These cases indicate the Denoised News Encoder’s efficacy in discerning and quantifying the potential influence of news on respective stock prices.

Secondly, concerning the causal graph discovered by Lag-dependent TCD, we denote the causal strength graph as the dot product of the causal graph $\bm{G}$ and the causal weight graph $\bm{\hat{G}}$. Every item of causal strength graph indicates not only the causality of two stocks but also the degree of causality. The visualized causal strength matrix for ACL18 is shown in Figure 3(b) with a heatmap. From various industries, we select companies with the highest and lowest market value. The top half of Figure 3(b) represents stocks corresponding to the nine companies with the largest market value, while the bottom half illustrates stocks from companies with the smallest. The causal strength of stocks is determined based on the average overall lags. In this heatmap, we could observe that distinct patterns emerge according to different market values. Stocks of low-market-value companies appear to have less pronounced causal relationships. We could also observe causal connections between certain high and low-market-value stocks. This is attributable to the dominant roles of large-value companies with their significant impact on those small-value firms and the stock prices.

Based on these observations, we compute the Spearman’s rank correlation coefficient [36] between the aforementioned company’s market value and their stock’s causal strength on ACL18, CMIN-CN, NI225, and FSTE100 datasets, representing the US, Chinese, Japanese, and UK stock markets respectively. The correlation results are shown in Appendix D and we also visualize some results in Figure 3(a). These results show a strong positive correlation between the market value and causal influence. This aligns with the intuition that not only do large-value companies hold pivotal economic positions, but also play crucial roles in influencing other companies. Our findings demonstrate that CausalStock does well in uncovering the causal relations within the stock market.

Following prior works [44, 23], we evaluate CausalStock’s applicability to the real world trading scenario. We conduct a portfolio strategy by choosing the top three stocks (based on predicted probabilities) with equal weight on each day of the test set and calculate the Accumulated Portfolio Value (APV) and Sharpe Ratio (SR) for evaluation. See Appendix  C.2 for a metrics details. The results on three datasets are shown in Table 4, which indicates that CausalStock achieves higher profits, and the excellent capabilities of CausalStock to balance risk with returns.