Theoretically Motivated Data Augmentation and Regularization for Portfolio Construction

There is an increasing interest in applying machine learning methods to problems in the finance industry. This trend has been expected for almost forty years (Fama,, 1970), when well-documented and fine-grained (minute-level) data of stock market prices became available. In fact, the essence of modern finance is fast and accurate large-scale data analysis (Goodhart and O’Hara,, 1997), and it is hard to imagine that machine learning should not play an increasingly crucial role in this field. In contemporary research, the central theme in machine-learning based finance is to apply existing deep learning models to financial time-series prediction problems (Imajo et al.,, 2020; Buehler et al.,, 2019; Jay et al.,, 2020; Imaki et al.,, 2021; Jiang et al.,, 2017; Fons et al.,, 2020; Lim et al.,, 2019; Zhang et al.,, 2020), which have demonstrated the hypothesized usefulness of deep learning for the financial industry.

However, one major existing gap in this interdisciplinary field of deep-learning finance is the lack of a theory relevant to justify finance-oriented algorithm design. The goal of this work is to propose such a framework, where machine learning practices are analyzed in a traditional financial-economic utility theory setting. Our theory implies that a simple theoretically motivated data augmentation technique is better for the task portfolio construction than not injecting any noise at all or some naive noise injection methods that have no theoretical justification. To summarize, our main contributions are (1) to demonstrate how we can use utility theory to analyze practices of deep-learning-based finance, and (2) to theoretically study the role of data augmentation in the deep-learning-based portfolio construction problem. Organization: the next section discusses the main related works; Section 3 provides the requisite finance background for understanding this work; Section 4 presents our theoretical contributions, which is a framework for understanding machine-learning practices in the portfolio construction problem; Section 5 describes how to practically implement the theoretically motivated algorithm; section 6 validates the theory with experiments on toy and real data.

Existing deep learning finance methods. In recent years, various empirical approaches to apply state-of-the-art deep learning methods to finance have been proposed (Imajo et al.,, 2020; Ito et al.,, 2020; Buehler et al.,, 2019; Jay et al.,, 2020; Imaki et al.,, 2021; Jiang et al.,, 2017; Fons et al.,, 2020). The interested readers are referred to (Ozbayoglu et al.,, 2020) for detailed descriptions of existing works. However, we notice that one crucial gap is the complete lack of theoretical analysis or motivation in this interdisciplinary field of AI-finance. This work makes one initial step to bridge this gap. One theme of this work is that finance-oriented prior knowledge and inductive bias is required to design the relevant algorithms. For example, Ziyin et al., (2020) shows that incorporating prior knowledge into architecture design is key to the success of neural networks and applied neural networks with periodic activation functions to the problem of financial index prediction. Imaki et al., (2021) shows how to incorporate no-transaction prior knowledge into network architecture design when transaction cost is incorporated.

In fact, most generic and popular machine learning techniques are proposed and have been tested for standard ML tasks such as image classification or language processing. Directly applying the ML methods that work for image tasks is unlikely to work well for financial tasks, where the nature of the data is different. See Figure 1, where we show the performance of a neural network directly trained to maximize wealth return on MSFT during 2019-2020. Using popular, generic deep learning techniques such as weight decay or dropout does not result in any improvement over the baseline. In contrast, our theoretically motivated method does. Combining the proposed method with weight decay has the potential to improve the performance a little further, but the improvement is much lesser than the improvement of using the proposed method over the baseline. This implies that a generic machine learning method is unlikely to capture well the inductive biases required to tackle a financial task. The present work proposes to fill this gap by showing how finance knowledge can be incorporated into algorithm design.

Data augmentation. Consider a training loss function of the additive form $L=\frac{1}{N}\sum_{i}\ell(x_{i},y_{i})$ for $N$ pairs of training data points $\{(x_{i},y_{i})\}_{i=1}^{N}$. Data augmentation amounts to defining an underlying data-dependent distribution and generating new data points stochastically from this underlying distribution. A general way to define data augmentation is to start with a datum-level training loss and transform it to an expectation over an augmentation distribution $P\left(z|(x_{i},y_{i})\right)$ (Dao et al.,, 2019), $\ell(x_{i},y_{i})\to\mathbb{E}_{(z_{i},g_{i})\sim P(z,g|(x_{i},y_{i}))}[\ell(z_{i},g_{i})]$, and the total training loss function becomes

One common example of data augmentation is injecting isotropic gaussian noise to the input (Shorten and Khoshgoftaar,, 2019; Fons et al.,, 2020), which is equivalent to setting $P(z,g|(x_{i},y_{i}))\sim\delta(g-y_{i})\exp\left[-(z-x_{i})^{\rm T}(z-x_{i})/(2\sigma^{2})\right]$ for some specified strength $\sigma^{2}$. Despite the ubiquity of data augmentation in deep learning, existing works are often empirical in nature (Fons et al.,, 2020; Zhong et al.,, 2020; Shorten and Khoshgoftaar,, 2019; Antoniou et al.,, 2017). For a relevant example, Fons et al., (2020) empirically evaluates the effect of different types of data augmentation in a financial series prediction task. Dao et al., (2019) is one major recent theoretical work that tries to understand modern data augmentation theoretically; it shows that data augmentation is approximately learning in a special kernel. He et al., (2019) argues that data augmentation can be seen as an effective regularization. However, no theoretically motivated data augmentation method for finance exists yet. One major challenge and achievement of this work is to develop a theory that bridges the traditional finance theory and machine learning methods. In the next section, we introduce the portfolio theory.

How to make optimal investment in a financial market is the central concern of portfolio theory. One unfamiliar with the portfolio theory may easily confuse the task the portfolio construction with wealth maximization trading or future price prediction. Before we introduce the portfolio theory, we first stress that the task of portfolio construction is not equivalent to wealth maximization or accurate price prediction. One can construct an optimal portfolio without predicting the price or maximizing the wealth increase.

Consider a market with an equity (a stock) and a fixed-interest rate bond (a government bond). We denote the price of the equity at time step $t$ as $S_{t}$, and the price return is defined as $r_{t}=\frac{S_{t+1}-S_{t}}{S_{t}}$, which is a random variable with variance $C_{t}$, and the expected return $g_{t}:=\mathbb{E}[r_{t}]$. Our wealth at time step $t$ is $W_{t}=M_{t}+n_{t}S_{t}$, where $M_{t}$ is the amount of cash we hold, and $n_{i}$ the shares of stock we hold for the $i$-th stock. As in the standard finance literature, we assume that the shares are infinitely divisible. Usually, a positive $n$ denotes holding (long) and a negative $n$ denotes borrowing (short). The wealth we hold initially is $W_{0}>0$, and we would like to invest our money on the equity. We denote the relative value of the stock we hold as $\pi_{t}=\frac{n_{t}S_{t}}{W_{t}}$. $\pi$ is called a portfolio. The central challenge in portfolio theory is to find the best $\pi$. At time $t$, our wealth is $W_{t}$; after one time step, our wealth changes due to a change in the price of the stock (setting the interest rate to be $0$): $\Delta W_{t}:=W_{t+1}-W_{t}=W_{t}\pi_{t}r_{t}$. The goal is to maximize the wealth return $G_{t}:=\pi_{t}\cdot r_{t}$ at every time step while minimizing risk²²2It is important to not to confuse the price return $r_{t}$ with the wealth return $G_{t}$.. The risk is defined as the variance of the wealth change:

The standard way to control risk is to introduce a “risk regularizer” that punishes the portfolios with a large risk (Markowitz,, 1959; Rubinstein,, 2002).³³3In principle, any concave function in $G_{t}$ can be a risk regularizer from classical economic theory (Von Neumann and Morgenstern,, 1947). One common alternative would be $R(G)=\log(G)$ (Kelly Jr,, 2011), and our framework can be easily extended to such cases. Introducing a parameter $\lambda$ for the strength of regularization (the factor of $1/2$ appears for convention), we can now write down our objective:

Here, $U$ stands for the utility function; $\lambda$ can be set to be the desired level of risk-aversion. When $g_{t}$ and $C_{t}$ is known, this problem can be explicitly solved. However, one main problem in finance is that its data is highly limited and we only observe one particular realized data trajectory, and $g_{t}$ and $C_{t}$ are hard to estimate. This fact motivates for the necessity of data augmentation and synthetic data generation in finance (Assefa,, 2020). In this paper, we treat the case where there is only one asset to trade in the market, and the task of utility maximization amounts to finding the best balance between cash-holding and investment. The equity we are treating is allowed to be a weighted combination of multiple stocks (a portfolio of some public fund manager, for example), and so our formalism is not limited to single-stock situations. In section C.1, we discuss portfolio theory with multiple stocks.

Recent advances have shown that the financial objectives can be interpreted as training losses for an appropriately inserted neural-network model (Ziyin et al.,, 2019; Buehler et al.,, 2019). It should come as no surprise that the utility function (3) can be interpreted as a loss function. When the goal is portfolio construction, we parametrize the portfolio $\pi_{t}=\pi_{\mathbf{w}}(x_{t})$ by a neural network with weights $\mathbf{w}$, and the utility maximization problem becomes a maximization problem over the weights of the neural network. The time-dependence is modeled through the input to the network $x_{t}$, which possibly consists of the available information at time $t$ for determining the future price.⁴⁴4It is helpful to imagine $x_{t}$ as, for example, the prices of the stocks in the past $10$ days. The objective function (to be maximized) plus a pre-specified data augmentation transform $x_{t}\to z_{t}$ with underlying distribution $p(z|x_{t})$ is then

where $\mathbb{E}_{t}:=\mathbb{E}_{z_{t}\sim p(z|x_{t})}$. In this work, we abstract away the details of the neural network to approximate $\pi$. We instead focus on studying the maximizers of this equation, which is a suitable choice when the underlying model is a neural network because one primary motivation for using neural networks in finance is that they are universal approximators and are often expected to find such maximizers (Buehler et al.,, 2019; Imaki et al.,, 2021).

The ultimate financial goal is to construct $\pi^{*}$ such that the utility function is maximized with respect to the true underlying distribution of $S_{t}$, which can be used as the generalization loss (to be maximized):

Note the difference in taking the expectation between Eq (4) and (5) is that $\mathbb{E}_{t}$ is computed with respect to the training set we hold, while $\mathbb{E}_{S_{t}}:=\mathbb{E}_{S_{t}\sim p(S_{t})}$ is computed with respect to the underlying distribution of $S_{t}$ given its previous prices. We used the same short-hands for $\text{Var}_{t}$ and $\text{Var}_{S_{t}}$. Technically, the true utility we defined is an in-sample counterfactual objective, which roughly evaluates the expected utility to be obtained if we restart from yesterday, which is a relevant measure for financial decision making. In Section 4.5, we also analyze the out-of-sample performance when the portfolio is static.

Our results strongly motivate for a specially designed data augmentation for financial data. For a data point consisting purely of past prices $(S_{t},...,S_{t+L},S_{t+L+1})$ and the associated returns $(r_{t},...,r_{t+L-1},r_{t+L})$, we use $x=(S_{t},...,S_{t+L})$ as the input for our model $f$, possibly a neural network, and use $S_{t+L+1}$ as the unseen future price for computing the training loss. Our results suggests that we should randomly noisify both the input $x$ and $S_{t+L+1}$ at every training step by

where $\epsilon_{i}$ are i.i.d. samples from $\mathcal{N}(0,1)$, and $c$ is a hyperparameter to be tuned. While the theory suggests that $c$ should be $1/2$, it is better to make it a tunable-parameter in algorithm design for better flexibility; $\hat{\sigma}_{t}$ is the instantaneous volatility, which can be estimated using standard methods in finance (Degiannakis and Floros,, 2015). One might also assume $\hat{\sigma}$ into $c$.

We validate our theoretical claim that using multiplicative noise with strength $\sqrt{r}$ is better than not using any data augmentation or using a data augmentation that is not suitable for the nature of portfolio construction (such as an additive Gaussian noise). We emphasize that the purpose of this section is for demonstrating the relevance of our theory to real financial problems, not for establishing the proposed method as a strong competitive method in the industry. We start with a toy dataset that follows the theoretical assumptions and then move on to real data with S&P500 prices. The detailed experimental settings are given in Section A. Unless otherwise specified, we use a feedforward neural network with the number of neurons $10\to 64\to 64\to 1$ with ReLU activations. Training proceeds with the Adam optimizer with a minibatch size of $64$ for 100 epochs with the default parameter settings.⁵⁵5In our initial experiments, we also experimented with different architectures (different depth or width of the FNN, RNN, LSTM), and our conclusion that the proposed augmentation outperforms the specified baselines remain unchanged.

We use the Sharpe ratio as the performance metric (the larger the better). Sharpe ratio is defined as $SR_{t}=\frac{\mathbb{E}[\Delta W_{t}]}{\sqrt{\text{Var}[\Delta W_{t}]}}$, which is a measure of the profitability per risk. We choose this metric because, in the framework of portfolio theory, it is the only theoretically motivated metric of success (Sharpe,, 1966). In particular, our theory is based on the maximization of the mean-variance utility in Eq. (3) and it is well-known that the maximization of the mean-variance utility is equivalent to the maximization of the Sharpe ratio. In fact, it is a classical result in classical financial research that all optimal strategies must have the same Sharpe ratio (Sharpe,, 1964) (also called the efficient capital frontier). For the synthetic tasks, we can generate arbitrarily many test points to compare the Sharpe ratios unambiguously. We then move to experiments on real stock price series; the limitation is that the Sharpe ratio needs to be estimated and involves one additional source of uncertainty.⁶⁶6We caution the readers to not to confuse the problem of portfolio construction with the problem of financial price prediction. Portfolio construction is the primary focus of our work and is fundamentally different from the problem of financial price prediction. Our method is not relevant and cannot be used directly for predicting future prices. As in real life, one does not need to be able to predict prices to decide which stock to purchase.

In this work, we have presented a theoretical framework relevant to finance and machine learning to understand and analyze methods related to deep-learning-based finance. The result is a machine learning algorithm incorporating prior knowledge about the underlying financial processes. The good performance of the proposed method agrees with the standard expectation in machine learning that performance can be improved if the right inductive biases are incorporated. We have thus succeeded in showing that building machine learning algorithms that is rooted firmly in financial theories can have a considerable and yet-to-be achieved benefit. We hope that our work can help motivating for more works that approaches the theoretical aspects of machine learning algorithms that are used for finance.

The limitation of the present work is obvious; we only considered the kinds of data augmentation that takes the form of noise injection. Other kinds of data augmentation may also be useful to the finance; for example, (Fons et al.,, 2020) empirically finds that magnify (Um et al.,, 2017), time warp (Kamycki et al.,, 2020), and SPAWNER (Le Guennec et al.,, 2016) are helpful for financial series prediction, and it is interesting to apply our theoretical framework to analyze these methdos as well; a correct theoretical analysis of these methods is likely to advance both the deep-learning based techniques for finance and our fundamental understanding of the underlying financial and economic mechanisms. Meanwhile, our understanding of the underlying financial dynamics is also rapidly advancing; we foresee better methods to be designed, and it is likely that the proposed method will be replaced by better algorithms soon. There is potentially positive social effects of this work because it is widely believed that designing better financial prediction methods can make the economy more efficient by eliminating arbitrage (Fama,, 1970); the cautionary note is that this work is only for the purpose of academic research, and should not be taken as an advice for monetary investment, and the readers should evaluate their own risk when applying the proposed method.