Statistical Proxy based Mean-Reverting Portfolios with Sparsity and Volatility Constraints

Portfolios exhibiting mean-reverting behavior are of great interest to practitioners due to their predictability property that creates arbitrage opportunities. The task is to construct a portfolio that tends to return to its average value over time and use its performance to make trading decisions. Traditionally such portfolios are obtained by considering a linear combination of assets that are stationary through co-integration methods. However, such an approach often results in a large number of assets, which may not own reasonable volatility; making such portfolios practically inapplicable [12, 28]. Therefore, along with mean-reversion, reasonable volatility is a mandatory property in realistic situations. Moreover, trading costs can be notably reduced by limiting the number of assets, and hence sparse portfolios offer advantages (note that the notion of sparsity is widely used in signal processing and machine learning applications; e.g., [19] and references therein). Hence, constructing a mean-reverting portfolio exhibiting reasonable volatility and comprising a relatively smaller number of assets has found attention in the literature [20, 8]. Recognize that the concept of sparsity holds significant relevance across various applications within contemporary science [18, 24, 17, 1].

Three standard proxies that reflect the mean-reversion property are the predictability measure, the portmanteau criterion, and crossing statistics [9]. The predictability measure, initially defined for stationary processes in [4] and later generalized for non-stationary processes by [3], assesses how similar a time series is to a noise process. Let $A_{i}$’s for $i=0,1,\dots,q$ be empirical autocovariance matrices related to a multivariate time series (see (16) for details), then the mean-reversion property via the predictability proxy can be captured using the following problem [9]:

The portmanteau statistic, proposed by [16], is used to determine if a process is a white noise. In particular, mean reversion is obtained via portamento criterion proxy as follows [9]:

The choice of $q$ depends on factors such as data availability, portfolio size, risk tolerance, and computational complexity; hence, in applications, sensitivity analysis is recommended for selecting an appropriate $q$ [11, 14]. The crossing statistic calculates the likelihood of a process crossing its mean per unit of time [26]. Precisely, the crossing statistics proxy results in mean reversion by [9]:

$\gamma$ serves as a hyperparameter, governing the influence of lag-$i$ autocovariance matrices (see (16)) on the mean reversion characteristic of the target portfolio. On the other hand, recall that sufficient volatility can be obtained by imposing a constraint on the variance, that is, $xA_{0}x\geq\phi$ for some positive threshold $\phi$, where $A_{0}$ is the covariance matrix. Hence, statistical proxy-based problems (1), (2), and (3) together with volatility and sparsity constraints can be formulated into the following comprehensive model:

where $\alpha\in\{0,1\},2\leq q\in\mathbb{N},A_{0}\succ 0,A_{i}\succeq 0$ for $i=1,\dots,q,\gamma\geq 0,\phi>0$, and $k\leq n$ (for further details see 17)). We emphasize that autocovariance matrices provide information about the temporal dependence structure among variables in a time series data set and are commonly used in time series analysis and econometrics for forecasting, model estimation, and inference; see Section 3 for details. Lastly, it is important to emphasize that $\alpha$ is not a hyperparameter; instead, it is a parameter that is determined immediately by the model choice, as detailed in the equation (17). Also, the hyperparameter $\gamma$ only matters when we consider the crossing statistics-based model and in fact, does not play a role in predictability- and portmanteau criterion-based models.

Despite the extensive literature on solving close problems in portfolio optimization [7, 13, 15, 23], the only approach for handling ($P$) suggests to apply the following semidefinite (SDP) relaxation [8]:

with $\beta>0$ and $\|X\|_{1}:=\sum_{i,j}|X_{i,j}|$. However, this method exhibits several drawbacks: (i) optimal values of ($P$) and ($SDP-P$) may differ, (ii) a solution of ($SDP-P$) is not necessarily rank-one, and (iii) even if it is rank-one, a common approach is to apply sparse principal component analysis to it but, the qualitative properties of a solution of the sparse component analysis is not well-understood with respect to the original problem. Therefore, the effectiveness of applying an SDP relaxation to ($P$) lacks rigorous theoretical support. The main reason is that the presence of $\|X\|_{1}$ in the objective function of ($SDP-P$) breaks down the standard techniques that result in a relationship between a QCQP (without a cardinality constraint) with its original SDP relaxation. More specifically, there are no known theoretical results that specify the relationship between the optimal values and points of a solution of a QCQP that has a cordiality constraint with its SDP relaxation. Further, there exists a body of literature addressing general cardinality problems through other relaxation techniques like mixed-integer nonlinear programming [2], and continuous relaxations of mixed-integer programs [6] but such methods are less promising because they are not using the interesting structure of (P) in their design.

Therefore, we propose an efficient tailored algorithm that exploits the structure of (P) and utilizes a penalty decomposition method to directly solve ($P$) which obtains a KKT point of ($P$). To the best of our knowledge, the proposed algorithm is the first specialized approach specifically designed to directly tackle the problem ($P$) based on the portmanteau criterion and crossing statistics proxies. This unifying algorithm not only is applicable to all three proxies mentioned above but also works for nonconvex objective functions. In the proposed methodology, we use the variable splitting technique to simplify our complex problem ($P$) that involves highly nonlinearly coupled constraints. Generally, the splitting approach involves introducing additional variables to the problem, which helps loosen the coupling between the original variables [27]. This transformation enables the problem to be tackled more efficiently. After splitting and penalizing over the related equality constraints, a sequence of penalized subproblems is in hand that could be approximately solved. Each subproblem is efficiently tackled by a block coordinate algorithm that outputs a saddle point of the corresponding subproblem. Each step of the block coordinate algorithm either has a closed-form solution or is shown to be tractable; see problems ($P_{x}$), ($P_{y}$), and ($P_{z}$), respectively, in Section 2. In addition, the sequence of objective values in the block coordinate algorithm is either strictly decreasing or reaches equality at a finite step, which also yields a saddle point. We establish that our tailored penalty decomposition algorithm finds a KKT point of the original problem ($P$). Moreover, in the nonconvex case, the analysis of the algorithm can be simply extended to demonstrate that our algorithm is successful again, if the starting penalty parameter is larger than a finite positive bound and if the objective function has a bounded level set; see Theorem 2.3 in Section 5 and its proof 5.4 in the Appendix.

We mention that [20] introduces a penalty decomposition method that exclusively addresses the sparse, volatile, and mean-reverting problem associated with the predictability proxy, specifically for when $\alpha=1$ and $\gamma=0$ in ($P$). In this work, we extend this methodology to efficiently tackle problems arising from other proxies as well. We emphasize that, to the best of our knowledge, the portmanteau criterion- and crossing statistics-based problems with volatility and sparsity constraints have not been previously considered in the literature, even though they could produce more profitable portfolios as more information is leveraged by them. We numerically demonstrate that this is indeed the case for crossing statistics, which highlights the importance of designing an effective algorithm for this intractable problem; see Section 3. Further, our empirical results reveal the quadratic term in the objective function is crucial in capturing mean reversion.

We demonstrate the effectiveness of the algorithm by conducting a numerical comparison with the SDP relaxation approach using the S&P 500 dataset. Our method outperforms the SDP relaxation in terms of practical performance measures. Further, we provide a comparison of portfolio performance generated from predictability, portmanteau criterion, and crossing statistics proxies, when our algorithm is applied. The results show that after tuning the parameters $q$ and $\gamma$, the crossing statistics-based portfolio not only effectively captures the mean-reversion property, but also achieves superior performance in terms of Sharpe ratio and cumulative return and loss, compared to predictability and portmanteau criterion-based portfolios; see Section 3. This is consistent with the fact that its corresponding problem exploits more information from the data in this case. However, the portmanteau criterion-based portfolio practically may not be successful in retaining mean reversion, underscoring the significance of the quadratic term in the objective function for capturing this desirable property.

Organization. The remainder of the paper is organized as follows. Section 2 presents the proposed penalty decomposition algorithm for directly solving ($P$) together with all technical issues. Extensive numerical experiments are provided in 3, while Section 4 draws some concluding remarks. The proofs of the convergence analysis are presented in the Appendix.

Notation. The complement of a set $S$ is denoted as $S^{c}$ and its cardinality as $|S|$. For a natural number $n$, let $[n]:=\{1,2,\dots,n\}$. For $S=\{i_{1},i_{2}\dots,i_{|S|}\}\subseteq[n]$ and $x\in\mathbb{R}^{n},x_{S}\in\mathbb{R}^{n}$ is the coordinate projection of $x$ with respect to indices in $S$, that is, $(x_{S})_{i}=x_{i}$ for $i\in S$ and $(x_{S})_{i}=0$ for $i\in S^{c}$. By $\|.\|$, we mean the standard Euclidean norm. We show $A$ is positive semidefinite or definite by $A\succeq 0$ and $A\succ 0$, respectively. We denote the smallest and largest eigenvalues of $A$ with $\lambda_{\min}(A)$, and $\lambda_{\max}(A)$, respectively.

An algorithm for directly minimizing ($P$) that obtains a KKT point is presented next, with all proofs delegated to the Appendix. The PPC (standing for predictability, portmanteau criterion, and crossing statistics) based algorithm leverages a tailored penalty decomposition method in which a sequence of penalty subproblems is approximately solved. For each penalty subproblem, a block coordinate descent (BCD) algorithm is utilized that produces a saddle point of the penalty subproblem. The limit point of a suitable subsequence of such saddle points is demonstrated to be a KKT point of ($P$).

Here, we examine the performance of Algorithm 2 in finding sparse, volatile, and statistically proxy-based mean-reverting portfolios on real data coming from the US stock market Standard and Poor’s (S$\&$P 500) Index. This data set is often used for showing the practical effectiveness of an algorithm or for comparing numerical performance between competing approaches.

Recall that a statistical arbitrage strategy typically involves four main steps, namely constructing an appropriate asset pool, designing a mean-reverting portfolio, verifying the mean-reverting property through a unit-root test, and finally trading the favorable portfolio. The construction of an asset pool can be achieved using a method described in [8], which utilizes the smallest eigenvalue of the covariance matrix. In this paper, we focus on the crucial second step of developing a mean-reverting portfolio by minimizing predictability, portmanteau criterion, or crossing statistics measures, while incorporating volatility and sparsity constraints to ensure the profitability of the portfolio in practice. The mean-reversion property can be verified using a unit-root test such as the Dickey-Fuller test [10]. A comprehensive discussion on the trading strategy of a mean-reverting portfolio in [30, 29] is based on observed/normalized spread values. Normalizing the spread values makes the strategy less sensitive to absolute price levels. Table 1 in [29] provides examples of how the strategy works for different trading positions and normalized spread values. The strategy involves closing long positions and taking short positions when the normalized spread value exceeds a threshold, and not taking any action when the normalized spread value is negative. It is a simple yet effective strategy, but careful testing and analysis are necessary before adopting it as a long-term trading strategy.

In our numerical experiments, various standard performance metrics are used to measure the quality of portfolios, which in our case, are obtained from the limit point of Algorithm 2 and applying sparse PCA to the solution of (SDP-P). The first measure is called cumulative profit and loss ($P\&L$) is used to measure the overall return of a mean-reverting portfolio within a single trading period, from $t_{1}$ to $t_{2}$, and is calculated as

where $P\&L_{t}=\color[rgb]{0,0,0}{x}^{T}r_{t}(t-t_{o})-x^{T}r_{t-1}(t-1-t_{o})$ when a long position is opened, and $P\&L_{t}=\color[rgb]{0,0,0}x^{T}r_{t}(t-t_{o})-x^{T}r_{t-1}(t-1-t_{o})$ when a short position is opened at time $t_{o}$. For a particular asset, the value of $r_{t}(\tau)=\frac{p_{t}-p_{t-\tau}}{p_{t-\tau}}\approx\ln(p_{t})-\ln(p_{t-\tau})$, where $p_{t}$ represents the price of the asset at time $t$. Table 1 in [29] is used for this purpose with $d$ being the standard deviation of the portfolio. The Return on Investment (ROI) is another metric used to evaluate the investment return of a mean-reverting portfolio. It is calculated as follows: $\text{ROI}_{t}=\frac{P\&L_{t}}{\|x\|_{1}},$ where $P\&L_{t}$ is the profit and loss of the portfolio at time $t$, and $\|x\|_{1}$ is the L1-norm of the portfolio weights. The last metric used to evaluate the performance of a portfolio over a given period of time from $t_{1}$ to $t_{2}$ is the Sharpe ratio ($SR$), which is defined as

where $\mu_{ROI}=1/(t_{2}-t_{1})\sum_{t=t_{1}}^{t_{2}}ROI_{t}$, and $\sigma^{2}_{ROI}=1/(t_{2}-t_{1})\sum_{t=t_{1}}^{t_{2}}(ROI_{t}-\mu_{ROI})^{2}$. A portfolio with a higher $SR$ is considered more profitable.

To conduct the numerical experiments, we start by explaining how to select matrices $A_{i}$’s for $i\in\{0,1,\dots,q\}$. Given a multivariate stochastic process $\bf x=(\bf x_{t})_{t\in\mathbb{N}}$ with values in $\mathbb{R}^{n}$. For a nonnegative integer $s$, the lag-$s$ empirical autocovariance matrix of a sample path $x=(x_{1},x_{2},\dots,x_{T})$ of $\bf x_{t}$ is defined as

In particular, we have

We construct the asset pool by combining the pools suggested in optimization portfolio studies [8, 30, 29, 28], resulting in a pool of $n=30$ assets. The trading time period is from February 1st, 2012 to June 30th, 2014.

We run Algorithm 2 with $r=\sqrt{10},\epsilon_{I}=10^{-3},$ defined in (11) and $\epsilon_{O}=10^{-3}$, defined in (12), and the volatility threshold $\phi$ selected to be larger than thirty percent of the median variance of all assets in the pool (see [8]). In order to obtain reasonable results and compare the performance of the portmanteau criterion- and crossing statistics-based portfolios, we must decide upon a specific $q$. For this goal, we select the best $q$ among the candidate set $\{2,3,4\}$ when we apply our algorithm for the case of the portmanteau criterion as follows. We measure the average Sharpe ratios for a small, medium, and large sparsity level $k=5,10,$ and $17$. Based on this procedure (results not reported due to space considerations), we get that $q=3$ produces better results than $q=2$, but the differences between $q=4$ and $q=3$ are rather marginal. Hence, in our experiments, we set $q=3$. After fixing $q=3$, the next step involves selecting an appropriate $\gamma$. For this aim, we concentrate exclusively on portfolio optimization based on crossing statistics. We opt for a range of values within the interval $(0.0001,1]$, incrementally adjusting $\gamma$ with a step size of 0.0009. While applying Algorithm 2 to the crossing statistic-based model across various sparsity levels, specifically $k=5,10,and17$, we evaluate the average Sharpe ratios. By this process, the best candidate is realized as $\gamma=0.001$. Furthermore, for (SDP-P), we need to determine $\beta$ as well. To select this hyperparameter, we apply the methodology presented in [8] (that is, we solve this convex problem and apply sparse principal component analysis on its solution to introduce the corresponding portfolio), for all three cases reported in (17) (with $q=3$, and $\gamma=0.001$) and values from 0.001 to 1.999, with 0.009 increments. This grid search revealed that the range between 0.8 and 1.2 showed better promise for $\beta$ values. We then narrowed down our selection within this range, using a 0.01 step. Throughout this iterative approach, we found that $\beta=1$ yielded the highest Sharpe ratio. As such, $\beta$ is selected as 1 in our comparisons.

The spread pictures in Figures 1, 2, and 3 depict that the proposed method illustrates better mean reversion overall and the second and third pictures in these figures show that our method surpasses the SDP relaxation technique across both standard cumulative P&L and Sharpe ratio assessments. This holds true for a range of sparsity levels, including small, medium, and large, as well as across all three portfolio models: predictability-based, portmanteau criterion-based, and crossing statistics-based portfolios. Nevertheless, we should emphasize that the numerical performance of the SDP relaxation idea raises intriguing theoretical research questions that surely require thorough investigation. For example, under what conditions on the problem data in ($P$), its SDP relaxation ($SDP-P$) has a rank-one solution. And, if available, how to use a rank-one solution of ($SDP-P$) to construct a solution of ($P$).

Next, we compare the performance of portfolios generated from the predictability, portmanteau criterion, and crossing statistics proxies, based on Algorithm 2. Figure 4 demonstrates that after careful tuning of the parameters $q$ and $\gamma$, the crossing statistics-based portfolio not only accurately captures the mean-reversion property, but also achieves superior performance in terms of Sharpe ratio and cumulative return and loss, surpassing the predictability and portmanteau criterion-based portfolios. This observation completely aligns with the idea that Algorithm 2 leverages more information from the data in this case. In addition, the portmanteau criterion proxy seems to fail in retaining mean reversion, underscoring the critical role of the quadratic term in the objective function for capturing this desirable property.

The paper developed an algorithm that solves the underlying optimization problem of constructing portfolios of assets, having the mean-reverting property, and also being sparse and exhibiting reasonable volatility. Three main proxies for capturing mean-reversion are predictability, portmanteau criterion, and crossing statistics. A comprehensive optimization model was developed for this task. We leverage the variable splitting technique to unfold highly coupled nonconvex constraints of this model and propose a tailored penalty decomposition method, that solves a sequence of penalty subproblems via an efficient block coordinate method. We establish that not only the proposed algorithm finds a KKT point when the objective function is convex, but also its analysis can be extended to a nonconvex objective function (with partial modifications). We numerically examine the performance of the algorithm and show that it outperforms the SDP relaxation approach in terms of standard measures on an S&P 500 data set. Further, we compare these statistical proxy-based portfolios and demonstrate that, after tuning hyperparameters, the crossing statistics surrogate captures mean-reversion well and achieves better performance compared to predictability and portmanteau criterion-based portfolios. Our method has significant potential applications in finance, economics, and other related fields.

All technical proofs are given next.