This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

Market Directional Information Derived From (Time, Execution Price, Shares Traded) Sequence of Transactions.
On The Impact From The Future.

Vladislav Gennadievich Malyshkin [email protected] Ioffe Institute, Politekhnicheskaya 26, St Petersburg, 194021, Russia    Mikhail Gennadievich Belov [email protected] Lomonosov Moscow State University, Faculty of Mechanics and Mathematics, GSP-1, Moscow, Vorob’evy Gory, 119991, Russia
(September, 20, 2022)
Abstract
$Id: ImpactFromTheFuture.tex,v 1.269 2022/10/09 10:41:55 mal Exp $

An attempt to obtain market directional information from non–stationary solution of the dynamic equation: ‘‘future price tends to the value maximizing the number of shares traded per unit time’’ is presented. A remarkable feature of the approach is an automatic time scale selection. It is determined from the state of maximal execution flow calculated on past transactions. Both lagging and advancing prices are calculated.

preprint: V.M.

Времена Пугачёвского бунта. Самозванец выступает перед народом, говорит о грядущем счастье, которое придёт в форме мужицкого царства. Пленный офицер спрашивает: ‘‘Откуда деньги будут на всю эту благодать’’? Пугачёв ответил: ‘‘Ты что, дурак? Из казны жить будем!’’

Народная легенда, 1774.

I Introduction

Introduced in [1] the ultimate market dynamics problem: an evidence of existence (or a proof of non–existence) of an automated trading machine, consistently making positive P&L trading on a free market as an autonomous agent can be formulated in its weak and strong forms[2]: whether such an automated trading machine can exist with legally available data (weak form) and whether it can exist with transaction sequence triples (time, execution price, shares traded) as the only information available (strong form); in the later case execution flow I=dV/dtI=dV/dt is the only available characteristic determining market dynamics.

Let us formulate the problem in the third, ‘‘superstrong’’, form: Whether the future value of price can be predicted from (time, execution price, shares traded) sequence of past transactions? Previously[3, 4] we thought this is not possible, only P&L that includes not only price dynamics but also trader actions can be possibly predicted. Recent results changed our opinion.

There are two types of predicted price: ‘‘lagging’’ (retarded) and ‘‘advancing’’ (future) Lagging price PRetP^{Ret} corresponds to past observations; future direction is determined by the difference of last price PlastP^{last} and PRetP^{Ret}. An example of PRetP^{Ret} is moving average. A common problem with lagging price is that it typically assumes an existence of a time scale the PRetP^{Ret} is calculated with, what gives incorrect direction for market movements with time scales lower than the one of PRetP^{Ret}; however making the time scale too low creates a large amount of false signals. Advancing price PAdvP^{Adv} is predicting actual value of future price; the direction is determined by the difference of PAdvP^{Adv} and PlastP^{last}. The PAdvP^{Adv} is typically calculated from limit order book information, brokerage clients order flow timings, etc.

In this work both lagging and advancing prices are calculated from (time, execution price, shares traded) sequence of past transactions. The key element is to determine the state |ψ[IH]\Ket{\psi^{[IH]}} of maximal execution flow I=dV/dtI=dV/dt (eigenvalue problem (10)), as experiments show it’s importance for market dynamics. Found |ψ[IH]\Ket{\psi^{[IH]}} state automatically selects the time scale what makes the approach robust.

Found lagging price (49) is the price in |ψ[IH]\Ket{\psi^{[IH]}} state P[IH]P^{[IH]} plus trending term that suppresses false signals. The advancing price is obtained by considering density matrix state ρJIH\|\rho_{JIH}\| corresponding to the state ‘‘since |ψ[IH]\Ket{\psi^{[IH]}} till now’’ and experimentally observed fact that operators pdIdt\left\|p\frac{dI}{dt}\right\| and Idpdt\left\|I\frac{dp}{dt}\right\| have to be equal in ρJIH\|\rho_{JIH}\| state. This corresponds to the result of our previous works [3, 5]: execution flow I=dV/dtI=dV/dt (the number of shares traded per unit time), not trading volume VV (the number of shares traded), is the driving force of the market: asset price is much more sensitive to execution flow II (dynamic impact), rather than to traded volume VV (regular impact).

This paper is concerned only with obtaining directional information from a sequence of past transaction in a ‘‘single asset universe’’ just for simplicity, see Section VIII below for multi asset universe generalization. Whereas the dynamics theory of Section IV definitely requires additional research, the lagging indicator (49) of Section VI, see Fig. 8, can be practically applied to trading even in a single asset universe. In this work we do not implement any trading ideas of [3, 4], where a concept of liquidity deficit trading: open a position at low II, then close already opened position at high II, as this is the only strategy that avoids eventual catastrophic P&L losses. This paper is concerned only with obtaining a directional information that is required to determine what side the position has to be open on a liquidity deficit event.

II The State Of Maximal Execution Flow

Introduce a wavefunction ψ(x)\psi(x) as a linear combination of basis function Qk(x)Q_{k}(x):

ψ(x)\displaystyle\psi(x) =\displaystyle= k=0n1αkQk(x)\displaystyle\sum\limits_{k=0}^{n-1}\alpha_{k}Q_{k}(x) (1)

Then an observable market–related value ff, corresponding to probability density ψ2(x)\psi^{2}(x), is calculated by averaging timeserie sample with the weight dμ=ψ2(x(t))ω(t)dtd\mu=\psi^{2}(x(t))\omega(t)dt; the expression corresponds to an estimation of Radon–Nikodym derivative[6]:

fψ\displaystyle f_{\psi} =\displaystyle= ψ|f|ψψ|ψ\displaystyle\frac{\Braket{\psi}{f}{\psi}}{\Braket{\psi}{\psi}} (2)
fψ\displaystyle f_{\psi} =\displaystyle= j,k=0n1αjQj|f|Qkαkj,k=0n1αjQj|Qkαk\displaystyle\frac{\sum\limits_{j,k=0}^{n-1}\alpha_{j}\Braket{Q_{j}}{f}{Q_{k}}\alpha_{k}}{\sum\limits_{j,k=0}^{n-1}\alpha_{j}\Braket{Q_{j}}{Q_{k}}\alpha_{k}} (3)

For averages we use bra–ket notation by Paul Dirac: ψ|\Bra{\psi} and |ψ\Ket{\psi}. The (2) is plain ratio of two moving averages, but the weight is not regular decaying exponent ω(t)\omega(t) from (55), but exponent multiplied by wavefunction squared as dμ=ψ2(x(t))ω(t)dtd\mu=\psi^{2}(x(t))\omega(t)dt, the ψ2(x)\psi^{2}(x) defines how to average a timeserie sample. Any ψ(x)\psi(x) function is defined by nn coefficients αk\alpha_{k}, the value of an observable variable ff in ψ(x)\psi(x) state is a ratio of two quadratic forms on αk\alpha_{k} (3); as an example of a wavefunction see localized state (13), it can be used for Radon–Nikodym interpolation: f(y)ψy|f|ψy/ψy|ψyf(y)\approx\Braket{\psi_{y}}{f}{\psi_{y}}\Big{/}\Braket{\psi_{y}}{\psi_{y}}; familiar least squares interpolation is also available: f(y)ψy|fψy(y)=j,k=0n1QjfGjk1Qk(y)f(y)\approx\Braket{\psi_{y}}{f}\psi_{y}(y)=\sum_{j,k=0}^{n-1}\Braket{Q_{j}f}G^{-1}_{jk}Q_{k}(y).

One can also consider a more general form of average, dμ=P(x(t))ω(t)dtd\mu=P(x(t))\omega(t)dt, where P(x)P(x) is an arbitrary polynomial, not just the square of a wavefunction. These states correspond to a density matrix average:

fρP\displaystyle f_{\rho_{P}} =\displaystyle= Spurf|ρPSpurρP\displaystyle\frac{\mathrm{Spur}\left\|f\middle|\rho_{P}\right\|}{\mathrm{Spur}\left\|\rho_{P}\right\|} (4)

This average, the same as (2), is a ratio of two moving averages. For an algorithm to convert a polynomial P(x)P(x) to the density matrix ρP\|\rho_{P}\| see Theorem 3 of [7]. A useful application of the density matrix states is to study an average ‘‘since |ψ\Ket{\psi}’’; for example if |ψ\Ket{\psi} corresponds to a past dV/dtdV/dt spike, then the polynomial ‘‘since |ψ\Ket{\psi} till now’’ is P(x)=J(ψ2(x))P(x)=J(\psi^{2}(x)) with J()J(\cdot) defined in (61); price change between ‘‘now’’ and the time of spike is Plastψ|p|ψ=Spurdpdt|ρJ(ψ2)P^{last}-\Braket{\psi}{p}{\psi}=\mathrm{Spur}\left\|\frac{dp}{dt}\middle|\rho_{J(\psi^{2})}\right\|, similarly, total traded volume on this interval is SpurdVdt|ρJ(ψ2)\mathrm{Spur}\left\|\frac{dV}{dt}\middle|\rho_{J(\psi^{2})}\right\|.

The main idea of [3] is to consider a wavefunction (1) then to construct (3) quadratic forms ratio. A generalized eigenvalue problem can be considered with the two matrices from (3). The most general case corresponds to two operators AA and BB. Consider an eigenvalue problem with the matrices Qj|A|Qk\Braket{Q_{j}}{A}{Q_{k}} and Qj|B|Qk\Braket{Q_{j}}{B}{Q_{k}}:

|A|ψ[i]\displaystyle\Ket{A\middle|\psi^{[i]}} =λ[i]|B|ψ[i]\displaystyle=\lambda^{[i]}\Ket{B\middle|\psi^{[i]}} (5)
k=0n1Qj|A|Qkαk[i]\displaystyle\sum\limits_{k=0}^{n-1}\Braket{Q_{j}}{A}{Q_{k}}\alpha^{[i]}_{k} =λ[i]k=0n1Qj|B|Qkαk[i]\displaystyle=\lambda^{[i]}\sum\limits_{k=0}^{n-1}\Braket{Q_{j}}{B}{Q_{k}}\alpha^{[i]}_{k} (6)
ψ[i](x)\displaystyle\psi^{[i]}(x) =k=0n1αk[i]Qk(x)\displaystyle=\sum\limits_{k=0}^{n-1}\alpha^{[i]}_{k}Q_{k}(x) (7)
δij\displaystyle\delta_{ij} =ψ[i]|B|ψ[j]=k,m=0n1αk[i]Qk|B|Qmαm[j]\displaystyle=\Braket{\psi^{[i]}}{B}{\psi^{[j]}}=\sum\limits_{k,m=0}^{n-1}\alpha^{[i]}_{k}\Braket{Q_{k}}{B}{Q_{m}}\alpha^{[j]}_{m} (8)
λ[i]δij\displaystyle\lambda^{[i]}\delta_{ij} =ψ[i]|A|ψ[j]=k,m=0n1αk[i]Qk|A|Qmαm[j]\displaystyle=\Braket{\psi^{[i]}}{A}{\psi^{[j]}}=\sum\limits_{k,m=0}^{n-1}\alpha^{[i]}_{k}\Braket{Q_{k}}{A}{Q_{m}}\alpha^{[j]}_{m} (9)

If at least one of these two matrices is positively defined – the problem has a unique solution (within eigenvalues degeneracy). In the found basis |ψ[i]\Ket{\psi^{[i]}} the two matrices are simultaneously diagonal: (8) and (9). See (80) to convert an operator’s matrix from |ψ[i]\Ket{\psi^{[i]}} to QjQ_{j} basis and (81) to convert it from QjQ_{j} to |ψ[i]\Ket{\psi^{[i]}} basis.

In our previous work [3, 5, 1, 2] we considered various AA and BB operators, with the goal to find operators and states that are related to market dynamics. We established, that execution flow I=dV/dtI=dV/dt (the number of shares traded per unit time), not trading volume VV (the number of shares traded), is the driving force of the market: asset price is much more sensitive to execution flow II (dynamic impact), rather than to traded volume VV (regular impact). This corresponds to the matrices Qj|I|Qk=Qj|A|Qk\Braket{Q_{j}}{I}{Q_{k}}=\Braket{Q_{j}}{A}{Q_{k}} and Qj|Qk=Qj|B|Qk\Braket{Q_{j}}{Q_{k}}=\Braket{Q_{j}}{B}{Q_{k}}. These two matrices are volume- and time- averaged products of two basis functions. Generalized eigenvalue problem for operator I=dV/dtI=dV/dt is the equation to determine market dynamics:

|I|ψ[i]=λ[i]|ψ[i]\displaystyle\Ket{I\middle|\psi^{[i]}}=\lambda^{[i]}\Ket{\psi^{[i]}} (10)
k=0n1Qj|I|Qkαk[i]=λ[i]k=0n1Qj|Qkαk[i]\displaystyle\sum\limits_{k=0}^{n-1}\Braket{Q_{j}}{I}{Q_{k}}\alpha^{[i]}_{k}=\lambda^{[i]}\sum\limits_{k=0}^{n-1}\Braket{Q_{j}}{Q_{k}}\alpha^{[i]}_{k} (11)
ψ[i](x)\displaystyle\psi^{[i]}(x) =k=0n1αk[i]Qk(x)\displaystyle=\sum\limits_{k=0}^{n-1}\alpha^{[i]}_{k}Q_{k}(x) (12)
ψy(x)\displaystyle\psi_{y}(x) =i=0n1ψ[i](y)ψ[i](x)i=0n1[ψ[i](y)]2=i=0n1|ψ[i]ψy|ψ[i]=j,k=0n1Qj(x)Gjk1Qk(y)j,k=0n1Qj(y)Gjk1Qk(y)\displaystyle=\frac{\sum\limits_{i=0}^{n-1}\psi^{[i]}(y)\psi^{[i]}(x)}{\sqrt{\sum\limits_{i=0}^{n-1}\left[\psi^{[i]}(y)\right]^{2}}}=\sum\limits_{i=0}^{n-1}\Ket{\psi^{[i]}}\Braket{\psi_{y}}{\psi^{[i]}}=\frac{\sum\limits_{j,k=0}^{n-1}Q_{j}(x)G^{-1}_{jk}Q_{k}(y)}{\sqrt{\sum\limits_{j,k=0}^{n-1}Q_{j}(y)G^{-1}_{jk}Q_{k}(y)}} (13)
ψy|ψ[i]2\displaystyle\Braket{\psi_{y}}{\psi^{[i]}}^{2} =[ψ[i](y)ψy(y)]2=[ψ[i](y)]2k=0n1[ψ[k](y)]2\displaystyle=\left[\frac{\psi^{[i]}(y)}{\psi_{y}(y)}\right]^{2}=\frac{\left[\psi^{[i]}(y)\right]^{2}}{\sum\limits_{k=0}^{n-1}\left[\psi^{[k]}(y)\right]^{2}} (14)

The y=x0y=x_{0} is the time ‘‘now’’, ψy(x)\psi_{y}(x) is a wavefunction localized at x=yx=y. Here and below we write ψ0(x)\psi_{0}(x) instead of ψx0(x)\psi_{x_{0}}(x) to simplify notations. The ψ0|ψ[i]\Braket{\psi_{0}}{\psi^{[i]}} is the projection of the |ψ[i]\Ket{\psi^{[i]}} state of (10) eigenproblem to the state ‘‘now’’ |ψ0\Ket{\psi_{0}}.

Our analysis[3, 5, 1, 2] shows that among the states |ψ[i]\Ket{\psi^{[i]}} of the problem (10) the state corresponding to the maximal eigenvalue among all λ[i]\lambda^{[i]}, i=0n1i=0\dots n-1, is the most important for market dynamics. Consider various observable characteristics in this state |ψ[IH]\Ket{\psi^{[IH]}}.

Refer to caption
Figure 1: Price PP, price P[IH]P^{[IH]} (15), and maximal/minimal eigenvalues of (10) for AAPL stock on September, 20, 2012. The calculations in shifted Legendre basis with n=12n=12 and τ\tau=128sec. The execution flow eigenvalues are scaled and shifted to 693693 to fit the chart.

In Fig. 1 a demonstration of several observables: the price in |ψ[IH]\Ket{\psi^{[IH]}} state (15), maximal eigenvalue λ[IH]\lambda^{[IH]} of (10) problem, and minimal eigenvalue λ[IL]\lambda^{[IL]} (for completeness) are presented.

P[IH]\displaystyle P^{[IH]} =ψ[IH]|pI|ψ[IH]ψ[IH]|I|ψ[IH]\displaystyle=\frac{\Braket{\psi^{[IH]}}{pI}{\psi^{[IH]}}}{\Braket{\psi^{[IH]}}{I}{\psi^{[IH]}}} (15)

From these observable one can clearly see that singularities in II cause singularities in price, and that a change in |ψ[IH]\Ket{\psi^{[IH]}} localization causes an immediate ‘‘switch’’ in an observable. This switch is caused by the presence of n1n-1 internal degrees of freedom αk\alpha_{k} (nn coefficients, one less due to normalizing 1=ψ|ψ1=\Braket{\psi}{\psi}, Eq. (8)). Such a ‘‘switch’’ is not possible in regular moving average (16) since it has no any internal degree of freedom, hence, all regular moving average dependencies are smooth.

The state |ψ[IH]\Ket{\psi^{[IH]}} that maximizes the number of shares traded per unit time on past observations sample is the main result of our initial work [3].

III On Time Scale Selection of a Trading Strategy

Financial markets have no intrinsic time scales111 Trivial time scales such as seasonal, daily open/close, year end, etc. while actually do exist provide little trading opportunities. (at least those a market participant can take an advantage of). For US equity market — market timeserie data manifests an existence of time scales from microseconds to decades. For NASDAQ ITCH [8] data feed time-discretization is one nanosecond. Whereas real markets typically have no intrinsic time scale, any trading strategy typically does have an intrinsic time scale. This time scale is determined by: available data feeds, available execution, trader personal preferences, etc. An implementation of trading strategies with time scales under one second requires a costly IT infrastructure of data feed/execution, and is hard to program algorithmically; moreover, market liquidity at such a low time scale is low, a situation when a dozen of HFT firms are chasing a single limit order of 100 shares is very common. For trading strategies with a large time scale the major difficulty is that a trader, observing post-factum missed opportunities, often starts to ‘‘adjust’’ the strategy to lower time scales. For professional money managers (managing other people money), with the rare exception of ‘‘super-stars’’, the maximal possible time scale is one month: once a month a letter to investors explaining the fund performance is required to be sent. There is no such a ‘‘monthly’’ constraint for somebody managing his own money, for example, an individual crypto investor may be 50% down in April 2022 – but for him this problem is not as big as it were for a fund. For traders the most popular time scales are between ‘‘daily trading’’ and ‘‘monthly P&L’’; these time scales provide sufficient number of opportunity events along with market data availability (e.g. Bloomberg). Important that these time scales are ‘‘compatible’’ with human reaction time.

The major drawbacks of trading systems the authors observed among institutional investors/hedge funds/individual investors is that all of them typically have a few time scales. Most often – a single time scale. It may be explicit or implicit, but it almost always exists. The contradiction between a spectrum of time scales of the financial markets and a single time scale of a trading system is the most common limitation in trading systems design.

Refer to caption
Figure 2: An example of regular exponential moving average corresponding to τ=128\tau=128s and τ=256\tau=256s. Standard deviation is also calculated with the same τ\tau and moving average ±\pm standard deviation is plotted as a thin line in the same color. As τ\tau increases – the moving average ‘‘shifts to the right’’ (τ\tau-proportional time delay, lagging indicator). The data is for AAPL stock on September, 20, 2012.

Consider familiar demonstration with a moving average. Let PτP^{\tau} be a regular exponential moving average. The average \Braket{\cdot} is calculated with the weight (55):

Pτ(tnow)\displaystyle P^{\tau}(t_{now}) =pII=Q0pIQ0I=Q0|pI|Q0Q0|I|Q0=tnow𝑑Vω(t)p(t)tnow𝑑Vω(t)\displaystyle=\frac{\Braket{pI}}{\Braket{I}}=\frac{\Braket{Q_{0}pI}}{\Braket{Q_{0}I}}=\frac{\Braket{Q_{0}}{pI}{Q_{0}}}{\Braket{Q_{0}}{I}{Q_{0}}}=\frac{\int_{-\infty}^{t_{now}}dV\,\omega(t)p(t)}{\int_{-\infty}^{t_{now}}dV\,\omega(t)} (16)

The averaging dμ=ω(t)dtd\mu=\omega(t)dt takes place between the past and tnowt_{now} using exponentially decaying weight ω(t)=exp((tnowt)/τ)\omega(t)=\exp\left(-(t_{now}-t)/\tau\right). With τ\tau increase, the contributing to integral interval becomes larger and moving average ‘‘shifts to the right’’ (τ\tau-proportional time delay, lagging indicator). The (16) has no single parameter that can ‘‘adjust’’ the time scale as αk\alpha_{k} do in (3) where dμ=ψ2(x(t))ω(t)dtd\mu=\psi^{2}(x(t))\omega(t)dt. In (16) we have ψ(x)=const\psi(x)=const. Trading strategies that watch crossing between price and moving average, or between two moving averages calculated with different values of τ\tau, have the problem that the specific values of time scales are initially preset. We personally observed a number of successful (and failed) traders who were constantly watching moving averages on Bloomberg — we may tell that their success is caused by intuitively switching from one scale to another; if you ask such a person what he is doing – he cannot explain; but looking at him from a side it is clear – the person is trying to identify relevant time- and price- scales. Successful traders also jump frequently by observing assets of different classes; it is a common situation before placing a trade on GOOG to observe: DJI, AAPL, commodity, power generating industry, chemical industry – all withing less than a minute. If you go from a human (who select the time scale based on intuition, market knowledge, news, personal communications, experience, etc.) to an ‘‘automated trading machine’’ that has none of that – the problem of selecting the time scale becomes very difficult. The problem of automatic time scale selection is crucial in trading systems design. Another critically important problem is to adsorb information of different financial instruments. The theory presented below is perfectly applicable in multi asset universe; the analysis and interpretation, however, become more complicated, see Section VIII below for a discussion. In this paper we will be concerned only with a single asset universe to demonstrate the main ideas, and a detailed generalization of the theory to multi asset universe will be published elsewhere.

Whereas we still have no approach to price scale selection (the [5] uses price basis pkp^{k} as QkQ_{k}, but in the stationary case this is actually a time scale equivalent), we do have a practical method for an authomatic selection of the time scale.

Considered in Section II above the state |ψ[IH]\Ket{\psi^{[IH]}} that maximizes the number of shares traded per unit time on past observations sample determines the time scale. Let us consider in this state not the price and execution flow as we studied before, but simply time distance to ‘‘now’’ in |ψ[IH]\Ket{\psi^{[IH]}} state:

T[IH]\displaystyle T^{[IH]} =ψ[IH]|(tnowt)I|ψ[IH]ψ[IH]|I|ψ[IH]\displaystyle=\frac{\Braket{\psi^{[IH]}}{(t_{now}-t)I}{\psi^{[IH]}}}{\Braket{\psi^{[IH]}}{I}{\psi^{[IH]}}} (17)
Tτ\displaystyle T^{\tau} =(tnowt)II\displaystyle=\frac{\Braket{(t_{now}-t)I}}{\Braket{I}} (18)

here TτT^{\tau} is regular moving average. As all the values of time (future and past) are known, the (17) carry information about |ψ[IH]\Ket{\psi^{[IH]}} localization. When the value is small – a large dV/dtdV/dt spike event happened very recently. When it is large – a large spike happened a substantial time ago, the value is an information when a large spike in dV/dtdV/dt took place.

Refer to caption
Refer to caption
Figure 3: T[IH]T^{[IH]} and regular moving average (18) (dark blue) TτT^{\tau} for τ=128\tau=128s and τ=256\tau=256s; the values are multiplied by 10310^{-3} and shifted up to fit the chart. The AAPL stock on September, 20, 2012. The calculations in shifted Legendre basis with n=12n=12. Top: for operator I=dV/dtI=dV/dt. Bottom: for operator V/TV/T.

In Fig. 3 (top) the value of T[IH]T^{[IH]} (scaled by the factor 10310^{-3} and shifted up to fit the chart) is presented for τ=128\tau=128s and τ=256\tau=256s. One can clearly see that there is no smooth transition between the states, the ‘‘switch’’ happens instantly, there is no τ\tau-proportional time delay, what is typical for regular moving averages TτT^{\tau}. A linear dependence of T[IH]T^{[IH]} on time is also observed, this is an indication of stability of |ψ[IH]\Ket{\psi^{[IH]}} state identification. The value of T[IH]T^{[IH]} is the time scale; typically it is easier to work with the density matrix ρJ(ψ2)\rho_{J(\psi^{2})} obtained from ψ(x)=ψ[IH](x)\psi(x)=\psi^{[IH]}(x) rather than with the time scale itself; a typical operation with time scale – calculate an average of some observable in the interval of time scale length till ‘‘now’’: the density matrix does exactly this.

We have tried a number of other operator pairs in generalized eigenvalue problem (6)

AA BB eigenvalue meaning
Qj|I|Qk\Braket{Q_{j}}{I}{Q_{k}} Qj|Qk\Braket{Q_{j}}{Q_{k}} I=dV/dtI=dV/dt (execution flow)
Qj|V|Qk\Braket{Q_{j}}{V}{Q_{k}} Qj|T|Qk\Braket{Q_{j}}{T}{Q_{k}} V/TV/T (aggregated execution flow)
Qj|TIV|Qk\Braket{Q_{j}}{TI-V}{Q_{k}} Qj|T|Qk\Braket{Q_{j}}{T}{Q_{k}} IV/TI-V/T
Qj|V|Qk\Braket{Q_{j}}{V}{Q_{k}} Qj|Qk\Braket{Q_{j}}{Q_{k}} VV (traded volume)
Qj|p|Qk\Braket{Q_{j}}{p}{Q_{k}} Qj|Qk\Braket{Q_{j}}{Q_{k}} pp (price)
Qj|pI|Qk\Braket{Q_{j}}{pI}{Q_{k}} Qj|I|Qk\Braket{Q_{j}}{I}{Q_{k}} pp (price)
Qj|dpdt|Qk\Braket{Q_{j}}{\frac{dp}{dt}}{Q_{k}} Qj|Qk\Braket{Q_{j}}{Q_{k}} dp/dtdp/dt
Qj|dpdt|Qk\Braket{Q_{j}}{\frac{dp}{dt}}{Q_{k}} Qj|I|Qk\Braket{Q_{j}}{I}{Q_{k}} dp/dVdp/dV (market impact)
Qj|Plastp|Qk\Braket{Q_{j}}{P^{last}-p}{Q_{k}} Qj|V|Qk\Braket{Q_{j}}{V}{Q_{k}} PlastpV\frac{P^{last}-p}{V} (aggregated market impact)

among many others[1, 2]. An eigenproblem with an additional constraint was also considered, see ‘‘Appendix F’’ of [2] and, more generally, ‘‘Appendix G’’ of [6]. All price-related operators cause noisy behavior, no ‘‘switching’’ whatsoever. Only the operator V/TV/T does have similar to I=dV/dtI=dV/dt switching (but less pronounced); it is also more sensitive to τ\tau selection. Time to max spike in V/TV/T is presented in Fig. 3 (bottom). See [2] about the properties of |V|ψ=λ|T|ψ\Ket{V|\psi}=\lambda\Ket{T|\psi} states: ‘‘Appendix C: The state of maximal aggregated execution flow V/TV/T’’.

This makes us to conclude that the state to determine the time scale is the state |ψ[IH]\Ket{\psi^{[IH]}} that maximizes the number of shares traded per unit time on past observations sample. This state allows us to average an observable ff with the weights:

meaning measure f𝑑μ\int fd\mu
‘‘at spike’’ dμ=ψ[IH]2(x(t))ω(t)dtd\mu={\psi^{[IH]}}^{2}(x(t))\omega(t)dt ψ[IH]|f|ψ[IH]\Braket{\psi^{[IH]}}{f}{\psi^{[IH]}}
variation ‘‘at spike’’ dμ=2ED(ψ[IH])ψ[IH](x(t))ω(t)dtd\mu=2\,\mathrm{ED}\left(\psi^{[IH]}\right)\psi^{[IH]}(x(t))\omega(t)dt 2ED(ψ[IH])|f|ψ[IH]2\Braket{\mathrm{ED}\left(\psi^{[IH]}\right)}{f}{\psi^{[IH]}}
‘‘since spike till now’’ dμ=P(x(t))ω(t)dtd\mu=P(x(t))\omega(t)dt; P(x)=J(ψ[IH]2)P(x)=J\left({\psi^{[IH]}}^{2}\right) Spurf|ρP\mathrm{Spur}\|f|\rho_{P}\|
‘‘since since spike’’ dμ=P(x(t))ω(t)dtd\mu=P(x(t))\omega(t)dt; P(x)=J(J(ψ[IH]2))P(x)=J\left(J\left({\psi^{[IH]}}^{2}\right)\right) Spurf|ρP\mathrm{Spur}\|f|\rho_{P}\|

Found solution automatically adjusts averaging weight what makes the value of parameter τ\tau in (55) much less important. The ‘‘switch’’ happens instantly, without a τ\tau-proportional time delay as it were for a regular moving average.

IV On The Impact From The Future

The concept of the Impact From The Future was introduced in [1]. It predicts the value of future execution flow. Given currently observed (at t=tnowt=t_{now}) value of execution flow I0=ψ0|I|ψ0I_{0}=\Braket{\psi_{0}}{I}{\psi_{0}} we know with certainty that future value of execution flow I0FI_{0}^{F} will be greater than I0I_{0} because more trading will definitely occur in the future. But how to estimate the value of I0FI_{0}^{F}? The maximal eigenvalue λ[IH]\lambda^{[IH]} of (10) is used as the estimation of future execution flow I0FI_{0}^{F}:

I0F\displaystyle I_{0}^{F} =λ[IH]\displaystyle=\lambda^{[IH]} (19)
dIF\displaystyle dI^{F} =I0FI0\displaystyle=I_{0}^{F}-I_{0} (20)
dIF\displaystyle dI^{F} 0\displaystyle\geq 0 (21)

Whereas the I0I_{0} is an ‘‘impact from the past’’ (already observed current execution flow), the dIFdI^{F} is an ‘‘impact from the future’’ (not yet observed contribution to current execution flow); it’s value is non–negative by construction. Similar ideology (use past maximal value as an estimator of future value) is often applied by market practitioners to asset prices or their standard deviations. This is incorrect. Experimental observations show: this ideology is applicable only to execution flow I=dV/dtI=dV/dt, not to the trading volume, asset price standard deviation or any other observable.

A criterion of no information about the future can be formulated. If current I0I_{0} is close to λ[IH]\lambda^{[IH]}, this means that we have a ‘‘very dramatic market’’ right now and there is no information about the future of this market:

dIF\displaystyle dI^{F} =0\displaystyle=0 (22)

An alternative form of (22) is more convenient in practice because the value is [0:1][0:1] bounded:

ψ0|ψ[IH]2\displaystyle\Braket{\psi_{0}}{\psi^{[IH]}}^{2} =1\displaystyle=1 (23)

This means that |ψ[IH]\Ket{\psi^{[IH]}} and |ψ0\Ket{\psi_{0}} are the same (14). In practice a good value of the threshold is between [0.2:0.8][0.2:0.8] instead of the maximal value of 11. In Fig. 1 one can clearly see the spikes in λ[IH]\lambda^{[IH]} when (23) approaches 11, for I0(t)I_{0}(t) see Fig. 1 of [2], it is not presented in Fig. 1 to save the place.

We have the state |ψ[IH]\Ket{\psi^{[IH]}} and the criterion (23) of no information about the future. How to obtain directional information? Previously [3] we considered the price P[IH]P^{[IH]} (15) in the found state |ψ[IH]\Ket{\psi^{[IH]}} as an indicator related to market direction. The difference between PlastP^{last} and P[IH]P^{[IH]} was used as a directional indicator. A typical result is presented in Fig. 1. The price P[IH]P^{[IH]} is actually a moving average with positive weight having n1n-1 internal degrees of freedom. It determines the direction (and can possibly work for ‘‘reverse to the mean’’ type of strategy), but this is not the future price.

Consider a concept from classical machanics. Let us introduce a Lagrangian–like function:

𝒮\displaystyle{\cal S} =t1t2L(p,V,t)𝑑t\displaystyle=\int\limits_{t_{1}}^{t_{2}}L(p,V,t)dt (24)

and try to variate it. Let us first take LL to be an exact differential (e.g. total energy in classical mechanics L=T+UL=T+U). Then (24) carry no information about the dynamics, but we obtain two distinct terms t1t2T𝑑t\int_{t_{1}}^{t_{2}}Tdt and t1t2U𝑑t\int_{t_{1}}^{t_{2}}Udt that we can consider the difference of and obtain actual equation of motion. We implemented this strategy below by considering various hypothesises for action 𝒮\cal S and testing them experimentally.

IV.1 Volume Driven Dynamics

Assume that price changes are caused by trading volume. Introduce

L(p,V)\displaystyle L(p,V) =ddt(pPlast)(VVlast)\displaystyle=\frac{d}{dt}(p-P^{last})(V-V^{last})
=(VVlast)dpdt+(pPlast)dVdt\displaystyle=(V-V^{last})\frac{dp}{dt}+(p-P^{last})\frac{dV}{dt} (25)

Then ‘‘exact differential action’’

𝒮ed\displaystyle{\cal S}^{ed} =ψ[IH]missing|(VVlast)dpdt|ψ[IH]+ψ[IH]missing|(pPlast)I|ψ[IH]\displaystyle=\Braket{\psi^{[IH]}\left missing}{(V-V^{last})\frac{dp}{dt}}{\psi^{[IH]}}+\Braket{\psi^{[IH]}\left missing}{(p-P^{last})I}{\psi^{[IH]}} (26)

To obtain ‘‘actual’’ action we have to change the sign in between:

𝒮\displaystyle{\cal S} =ψ[IH]missing|(VVlast)dpdt|ψ[IH]ψ[IH]missing|(pPlast)I|ψ[IH]\displaystyle=\Braket{\psi^{[IH]}\left missing}{(V-V^{last})\frac{dp}{dt}}{\psi^{[IH]}}-\Braket{\psi^{[IH]}\left missing}{(p-P^{last})I}{\psi^{[IH]}} (27)

These two terms can be considered as ‘‘kinetic’’ and ‘‘potential’’ energy. It is difficult to variate (27) so let us just find the price PlastP^{last} that makes these two terms equal222Similar concept in mechanics corresponds to finding the state in which kinetic energy equals to potential energy. This gives an exact result for oscillators and approximate result for many other systems, see Virial theorem.:

ΔVD\displaystyle\Delta_{VD} =ψ[IH]missing|(VVlast)dpdt|ψ[IH]\displaystyle=\Braket{\psi^{[IH]}\left missing}{(V-V^{last})\frac{dp}{dt}}{\psi^{[IH]}} (28)
PEQ\displaystyle P^{EQ} =P[IH]1λ[IH]ΔVD\displaystyle=P^{[IH]}-\frac{1}{\lambda^{[IH]}}\Delta_{VD} (29)

The dynamics with (27) action is a volume-driven dynamics. Let I=dVdt=constI=\frac{dV}{dt}=const then both price and volume are linear functions on time, P=αt+CP=\alpha t+C and VVlast=ItV-V^{last}=It, and two terms in (27) are equal exactly in any state: (pPlast)I=(VVlast)dpdt(p-P^{last})I=(V-V^{last})\frac{dp}{dt}. Taking into account that |ψ[IH]\Ket{\psi^{[IH]}} is typically localized (see Fig. 6 of Ref. [2]) obtain ψ[IH]missing|(VVlast)dpdt|ψ[IH]ψ[IH]|VVlast|ψ[IH]ψ[IH]|dpdt|ψ[IH]\Braket{\psi^{[IH]}\left missing}{(V-V^{last})\frac{dp}{dt}}{\psi^{[IH]}}\approx\Braket{\psi^{[IH]}}{V-V^{last}}{\psi^{[IH]}}\Braket{\psi^{[IH]}}{\frac{dp}{dt}}{\psi^{[IH]}}. Since VVlastV-V^{last} is negative, the (29) means trend following, the trend is determined in |ψ[IH]\Ket{\psi^{[IH]}} state. In this state we have price equals to P[IH]P^{[IH]} and dp/dt=ψ[IH]|dpdt|ψ[IH]dp/dt=\Braket{\psi^{[IH]}}{\frac{dp}{dt}}{\psi^{[IH]}}. The (29) means that reference price will slowly follow the trend as trading proceed. This trend following stops only with |ψ[IH]\Ket{\psi^{[IH]}} state switch, what means a new spike in II has been observed and this new spike is now the ‘‘most dramatic market observed’’.

Refer to caption
Figure 4: Price PP, price P[IH]P^{[IH]} (15), PEQP^{EQ} (29) and maximal eigenvalue of (10) for AAPL stock on September, 20, 2012. The calculations in shifted Legendre basis with n=12n=12 and τ\tau=256sec. The execution flow eigenvalue is scaled and shifted to 693693 to fit the chart.

An important feature of PEQP^{EQ} is that in (29) there are only the moments that are calculated directly from sample using (66): QmI\Braket{Q_{m}I}, QmpI\Braket{Q_{m}pI}, and QmVdpdt\Braket{Q_{m}V\frac{dp}{dt}}. This makes all the calculations easy. In Fig. 4 we present the PEQP^{EQ} along with P[IH]P^{[IH]}. In [3] the best directional indicator found was the difference between last price and P[IH]P^{[IH]}, without trending term. As the price reaches some trading band – it starts crossing P[IH]P^{[IH]} multiple times thus creating false signals. The PEQP^{EQ} has a great advantage of extra trend following contribution in (29), what very much suppresses false signals. The result is also stable in situations when max II ‘‘switch’’ is missed.

IV.2 Execution Flow Driven Dynamics

Consider ‘‘exact differential’’ action with ψ[IH]missing|(pPlast)dVdt|ψ[IH]-\Braket{\psi^{[IH]}\left missing}{(p-P^{last})\frac{dV}{dt}}{\psi^{[IH]}} term from (25):

𝒮ed\displaystyle{\cal S}^{ed} =ψ[IH]missing|(Plastp)I|ψ[IH]=Spurddt(pPlast)I|ρJIH\displaystyle=\Braket{\psi^{[IH]}\left missing}{(P^{last}-p)I}{\psi^{[IH]}}=\mathrm{Spur}\left\|\frac{d}{dt}(p-P^{last})I\middle|\rho_{JIH}\right\|
=SpurIdpdt|ρJIH+SpurpdIdt|ρJIH\displaystyle=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|+\mathrm{Spur}\left\|p\frac{dI}{dt}\middle|\rho_{JIH}\right\| (30)

Then ‘‘actual’’ action333 Note that if one put in (31) ddt(VlastV)dpdt\left\|\frac{d}{dt}(V^{last}-V)\frac{dp}{dt}\right\| and ddt(Plastp)I\left\|\frac{d}{dt}(P^{last}-p)I\right\| instead of Idpdt\left\|I\frac{dp}{dt}\right\| and pdIdt\left\|p\frac{dI}{dt}\right\| the (27) is obtained. is considered as ‘‘kinetic’’ and ‘‘potential’’ terms split; the kinetic term is defined as the one with first derivative of price in Spur\mathrm{Spur} with ρJIH\left\|\rho_{JIH}\right\|.

𝒮\displaystyle{\cal S} =SpurIdpdt|ρJIHSpurpdIdt|ρJIH\displaystyle=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|p\frac{dI}{dt}\middle|\rho_{JIH}\right\| (31)

Here and below ρJIH\left\|\rho_{JIH}\right\| is the densitity matrix corresponding to the state ‘‘since |ψ[IH]\Ket{\psi^{[IH]}} till now’’ (60), obtained from the polynomial J(ψ[IH]2)J\left({\psi^{[IH]}}^{2}\right) (63) by applying Theorem 3 of [7], see \seqsplitcom/polytechnik/utils/BasisFunctionsMultipliable.java:getMomentsOfMeasureProducingPolynomialInKK_MQQM for numerical implementation; the ρJJIH\left\|\rho_{JJIH}\right\| is the densitity matrix corresponding to the polynomial J(J(ψ[IH]2))J\left(J\left({\psi^{[IH]}}^{2}\right)\right) (64);

ρJIH\displaystyle\left\|\rho_{JIH}\right\| =ρJ(ψ[IH]2)\displaystyle=\left\|\rho_{J\left({\psi^{[IH]}}^{2}\right)}\right\| (32)
ρJJIH\displaystyle\left\|\rho_{JJIH}\right\| =ρJ(J(ψ[IH]2))\displaystyle=\left\|\rho_{J\left(J\left({\psi^{[IH]}}^{2}\right)\right)}\right\| (33)

see \seqsplitcom/polytechnik/freemoney/IandDM.java:{QQDensityMatrix,QQDensityMatrix2} for numerical calculation of ρJIH\left\|\rho_{JIH}\right\| and ρJJIH\left\|\rho_{JJIH}\right\| from ψ[IH](x)\psi^{[IH]}(x). The second term in (31) does not depend on price shift pp+constp\to p+const as with (19) boundary condition we have SpurdIdt|ρJIH=0\mathrm{Spur}\left\|\frac{dI}{dt}\middle|\rho_{JIH}\right\|=0. The dynamics with (31) action is execution flow driven dynamics. Let I=dVdt=constI=\frac{dV}{dt}=const then the volume is a linear functions on time VVlast=ItV-V^{last}=It and the price is constant dp/dt=0dp/dt=0, both terms in (31) are zero in any state; moreover when p(t)=I(t)p(t)=I(t) the two terms are equal exactly for any I(t)I(t). This is different from the dynamics defined by (27) action where constant II causes linear dependence of price on time. With (31) action all changes in price are caused by changes in execution flow. Our previous observations [3, 5] show that asset prices are much more sensitive to execution flow II (dynamic impact), rather than to traded volume VV (regular impact).

Whereas the calculation of (27) action was easy because the moments were calculated directly from sample, the calculation of (31) is much more difficult. We can obtain directly from sample only operator pI\|pI\| and then, using (72), operator dpIdt=Idpdt+pdIdt\left\|\frac{dpI}{dt}\right\|=\left\|I\frac{dp}{dt}\right\|+\left\|p\frac{dI}{dt}\right\|:

IdpdtpdIdt\displaystyle\left\|I\frac{dp}{dt}\right\|-\left\|p\frac{dI}{dt}\right\| =2IdpdtdpIdt\displaystyle=2\left\|I\frac{dp}{dt}\right\|-\left\|\frac{dpI}{dt}\right\| (34)

The problem left is to calculate the operator Idpdt\left\|I\frac{dp}{dt}\right\|. Currently we do not have a method to obtain it exactly. The problem is simplified by the fact that we need not the operator Idpdt\left\|I\frac{dp}{dt}\right\| per se, but just SpurIdpdt|ρJIH\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|, what enables us to work with it’s approximation. See Appendix A below for several approximations for Idpdt\left\|I\frac{dp}{dt}\right\|; one can possibly try secondary sampling approach of Appendix D as an alternative route. Different approximations give noticeably different results. Nevetheless, assume we know the value of SpurIdpdt|ρJIH\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\| on past sample. Then, assume one more observation with price PEQP^{EQ} is coming. With the knowledge of future execution flow (19) we can put equal ‘‘kinetic’’ and ‘‘potential’’ terms in (31), thus to obtain the value of price PEQP^{EQ} at which (31) is zero (use (19) boundary condition with (72) expansion):

ΔI\displaystyle\Delta_{I} =SpurIdpdt|ρJIHSpurpdIdt|ρJIH\displaystyle=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|p\frac{dI}{dt}\middle|\rho_{JIH}\right\|
=2SpurIdpdt|ρJIHSpurdpIdt|ρJIH\displaystyle=2\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JIH}\right\| (35)
PEQ\displaystyle P^{EQ} =PlastΔIλ[IH]\displaystyle=P^{last}-\frac{\Delta_{I}}{\lambda^{[IH]}} (36)
=2PlastP[IH]2λ[IH]SpurIdpdt|ρJIH\displaystyle=2P^{last}-P^{[IH]}-\frac{2}{\lambda^{[IH]}}\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|

In (36) all ‘‘future’’ price contributions are moved to the left hand side and in the right hand side all the integration is performed till last observed point with p=Plastp=P^{last}. Technically (36) means: calculate the difference ΔI\Delta_{I} (35) on observed sample, and if it is not zero – the price will move on ΔI/λ[IH]-\Delta_{I}/\lambda^{[IH]} to compensate. Our experiments show that these two terms are very close to each other and the value of ΔI\Delta_{I} is small. One may also try other states, such as ρJJIH\left\|\rho_{JJIH}\right\| (33) to consider the operators in, but the property of operators Idpdt\left\|I\frac{dp}{dt}\right\| and pdIdt\left\|p\frac{dI}{dt}\right\| being equal in some density matrix state seems to be special to the state ρJIH\left\|\rho_{JIH}\right\|, see Appendix F below for a study of the state ρJJIH\left\|\rho_{JJIH}\right\|.

IV.3 Local Volume Driven Dynamics

Similarly to (30) one can consider the term ψ[IH]missing|(VVlast)dpdt|ψ[IH]-\Braket{\psi^{[IH]}\left missing}{(V-V^{last})\frac{dp}{dt}}{\psi^{[IH]}} from (25) to be an ‘‘exact differential’’ action:

𝒮ed\displaystyle{\cal S}^{ed} =ψ[IH]missing|(VlastV)dpdt|ψ[IH]=Spurddt(VVlast)dpdt|ρJIH\displaystyle=\Braket{\psi^{[IH]}\left missing}{(V^{last}-V)\frac{dp}{dt}}{\psi^{[IH]}}=\mathrm{Spur}\left\|\frac{d}{dt}(V-V^{last})\frac{dp}{dt}\middle|\rho_{JIH}\right\|
=SpurIdpdt|ρJIH+Spur(VVlast)d2pdt2|ρJIH\displaystyle=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|+\mathrm{Spur}\left\|(V-V^{last})\frac{d^{2}p}{dt^{2}}\middle|\rho_{JIH}\right\| (37)

Then ‘‘actual’’ action is:

𝒮\displaystyle{\cal S} =SpurIdpdt|ρJIHSpur(VVlast)d2pdt2|ρJIH\displaystyle=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|(V-V^{last})\frac{d^{2}p}{dt^{2}}\middle|\rho_{JIH}\right\| (38)

The dynamics with (38) action is local volume driven dynamics. Let I=dVdt=constI=\frac{dV}{dt}=const, then the volume is a linear functions on time VVlast=ItV-V^{last}=It. Two terms equal give quadratic dependence of price on time: P=αt2+CP=\alpha t^{2}+C; then two terms in (38) are equal exactly in any state: Idpdt=(VVlast)d2pdt2I\frac{dp}{dt}=(V-V^{last})\frac{d^{2}p}{dt^{2}}. There is a similar problem with calculation of the derivatives to the one considered above. Taking into account ddt(VVlast)dpdt=Idpdt+(VVlast)dp2dt2\left\|\frac{d}{dt}(V-V^{last})\frac{dp}{dt}\right\|=\left\|I\frac{dp}{dt}\right\|+\left\|(V-V^{last})\frac{dp^{2}}{dt^{2}}\right\| obtain:

Idpdt(VVlast)d2pdt2\displaystyle\left\|I\frac{dp}{dt}\right\|-\left\|(V-V^{last})\frac{d^{2}p}{dt^{2}}\right\| =2Idpdtddt(VVlast)dpdt\displaystyle=2\left\|I\frac{dp}{dt}\right\|-\left\|\frac{d}{dt}(V-V^{last})\frac{dp}{dt}\right\| (39)

As with (34) above the second term is obtained by applying (72) to the moments (VVlast)dpdt\left\|(V-V^{last})\frac{dp}{dt}\right\| that are available directly from sample. Consider one more observation with price PEQP^{EQ} coming. With the knowledge of future execution flow (19) we can obtain the equilibrium price:

ΔV\displaystyle\Delta_{V} =SpurIdpdt|ρJIHSpur(VVlast)d2pdt2|ρJIH\displaystyle=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|(V-V^{last})\frac{d^{2}p}{dt^{2}}\middle|\rho_{JIH}\right\|
=2SpurIdpdt|ρJIHSpurddt(VVlast)dpdt|ρJIH\displaystyle=2\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|\frac{d}{dt}(V-V^{last})\frac{dp}{dt}\middle|\rho_{JIH}\right\| (40)
PEQ\displaystyle P^{EQ} =PlastΔVλ[IH]\displaystyle=P^{last}-\frac{\Delta_{V}}{\lambda^{[IH]}} (41)

Technically (41) means: calculate the difference ΔV\Delta_{V} (40) on observed sample, and if it is not zero – the price will move on ΔV/λ[IH]-\Delta_{V}/\lambda^{[IH]} to compensate.

IV.4 Total Lagrangian Driven Dynamics

The same as in Sections IV.2 and IV.3 above we can consider the entire L(p,V)L(p,V) from (25) in |ψ[IH]\Ket{\psi^{[IH]}} state ψ[IH]|L(p,V)|ψ[IH]-\Braket{\psi^{[IH]}}{L(p,V)}{\psi^{[IH]}} to be an ‘‘exact differential’’ action:

𝒮ed\displaystyle{\cal S}^{ed} =ψ[IH]missing|(Plastp)I|ψ[IH]+ψ[IH]missing|(VlastV)dpdt|ψ[IH]\displaystyle=\Braket{\psi^{[IH]}\left missing}{(P^{last}-p)I}{\psi^{[IH]}}+\Braket{\psi^{[IH]}\left missing}{(V^{last}-V)\frac{dp}{dt}}{\psi^{[IH]}}
=2SpurIdpdt|ρJIH+SpurpdIdt|ρJIH+Spur(VVlast)d2pdt2|ρJIH\displaystyle=2\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|+\mathrm{Spur}\left\|p\frac{dI}{dt}\middle|\rho_{JIH}\right\|+\mathrm{Spur}\left\|(V-V^{last})\frac{d^{2}p}{dt^{2}}\middle|\rho_{JIH}\right\| (42)

This is actually the same expression as (26) above (with changed sign), but split differently into ‘‘kinetic’’ and ‘‘potential’’ energy terms. The ‘‘actual’’ action is then obtained by changing the sign of ‘‘potential’’ contributions:

𝒮\displaystyle{\cal S} =2SpurIdpdt|ρJIH(SpurpdIdt|ρJIH+Spur(VVlast)d2pdt2|ρJIH)\displaystyle=2\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\left\lgroup\mathrm{Spur}\left\|p\frac{dI}{dt}\middle|\rho_{JIH}\right\|+\mathrm{Spur}\left\|(V-V^{last})\frac{d^{2}p}{dt^{2}}\middle|\rho_{JIH}\right\|\right\rgroup (43)

The dynamics for I=dVdt=constI=\frac{dV}{dt}=const can be obtained in a regular way. The volume is a linear functions on time VVlast=ItV-V^{last}=It. The price with ‘‘kinetic’’ and ‘‘potential’’ terms equal gives cubic dependence of price on time: P=αt3+CP=\alpha t^{3}+C; the terms in (43) are equal exactly in any state: 2Idpdt=0+(VVlast)d2pdt22I\frac{dp}{dt}=0+(V-V^{last})\frac{d^{2}p}{dt^{2}}.

The ΔT\Delta_{T} correponds to the sum of (35) and (40):

ΔT\displaystyle\Delta_{T} =ΔI+ΔV\displaystyle=\Delta_{I}+\Delta_{V} (44)
=4SpurIdpdt|ρJIHSpurdpIdt|ρJIHSpurddt(VVlast)dpdt|ρJIH\displaystyle=4\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JIH}\right\|-\mathrm{Spur}\left\|\frac{d}{dt}(V-V^{last})\frac{dp}{dt}\middle|\rho_{JIH}\right\| (45)
PEQ\displaystyle P^{EQ} =PlastΔTλ[IH]\displaystyle=P^{last}-\frac{\Delta_{T}}{\lambda^{[IH]}} (46)

Whereas this ‘‘Total Lagrangian Driven’’ and ‘‘Volume Driven’’ dynamics of Section IV.1 above use the same ‘‘exact differential action’’: (42) and (26), they generate different dynamics as we differently split action into ‘‘kinetic’’ and ‘‘potential’’ terms. The dynamics of Section IV.1 is of trend following type, the direction changes only when |ψ[IH]\Ket{\psi^{[IH]}} ‘‘switches’’. The direction of ‘‘Total Lagrangian Driven’’ dynamics takes into account a number of factors in (43), thus it may reverse the direction even without a ‘‘switch’’ in |ψ[IH]\Ket{\psi^{[IH]}}.

V On Selection Of A Type Of The Dynamics

In Sections IV.1, IV.2, IV.3, and IV.4 we chose an ‘‘exact differential’’ action from which price dynamics was determined. The ‘‘volume driven’’ dynaimcs of Section IV.1 stays separately as it is a trend–following model with automatic time–scale selection of Section III, it is the simplest ‘‘practical’’ model. An important feature of possible ‘‘action’’ of the forms: (31), (38), and (43) is that all of them include fluctuations of execution flow dI/dtdI/dt. Now we have to choose which one corresponds to market dynamics most closely. Technically, we have three calculated characteristics: SpurdpIdt|ρJIH\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JIH}\right\| and Spurddt(VVlast)dpdt|ρJIH\mathrm{Spur}\left\|\frac{d}{dt}(V-V^{last})\frac{dp}{dt}\middle|\rho_{JIH}\right\| that are obtained exactly and SpurIdpdt|ρJIH\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\| that is obtain from an approximation, such as in the Appendix A. Now we need to select from them the most appropriate linear combination to obtain the future price according to (35), (40), or (44).

Refer to caption
Figure 5: Price PP, price P[IH]P^{[IH]} (15), and PEQP^{EQ} (35) for AAPL stock on September, 20, 2012. The ratio of aggregated VT\frac{V}{T} execution flow and λ[IH]\lambda^{[IH]} is also presented (shifted up to 693); it is about 1/21/2 in |ψ[IH]\Ket{\psi^{[IH]}} state. The calculations in shifted Legendre basis with n=12n=12 and τ\tau=256sec.

Experiments show that only execution flow driven dynamics of (31) form can possibly work to determine the future price. This type of dynamics has only two contributing terms: IdpdtI\frac{dp}{dt} and pdIdtp\frac{dI}{dt}, that are very close to each other in ρJIH\|\rho_{JIH}\| state. The sum of them is equal exactly to λ[IH](PlastP[IH])=SpurdpIdt|ρJIH\lambda^{[IH]}(P^{last}-P^{[IH]})=\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JIH}\right\| (the result of [3]), and their difference determines (35) future price (a new result of this paper). In Fig. (5) a demonstration of PEQP^{EQ} (35) (field \seqsplitcom/polytechnik/freemoney/PFuture.java:PEQ_I_fromDPI, obtained with Idpdt\|I\frac{dp}{dt}\| from (78) is presented. The result is almost identical to (88) approximation, field \seqsplitcom/polytechnik/freemoney/PFuture.java:PEQ_I_fromI2DtpDivI, and the ΔI\Delta_{I} is slightly lower with (77)). This is the only ‘‘advancing’’ (not lagging!) indicator we managed to obtain so far. A very important feature is that operators pdIdt\left\|p\frac{dI}{dt}\right\| and Idpdt\left\|I\frac{dp}{dt}\right\| from (34) are very close in ρJIH\rho_{JIH} state, and the difference determines future direction (35). This corresponds to 2λ[IH]SpurIdpdt|ρJIHSpurdpdt|ρJIH\frac{2}{\lambda^{[IH]}}\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\|\approx\mathrm{Spur}\left\|\frac{dp}{dt}\middle|\rho_{JIH}\right\|; the ratio of aggregated execution flow VT=SpurI|ρJIHSpurρJIH\frac{V}{T}=\frac{\mathrm{Spur}\left\|I\middle|\rho_{JIH}\right\|}{\mathrm{Spur}\left\|\rho_{JIH}\right\|} and execution flow λ[IH]\lambda^{[IH]} is about 0.50.5 in |ψ[IH]\Ket{\psi^{[IH]}} state, see green line in Fig. 5.

One may also consider other states, e.g. from [2]: ‘‘Appendix C: The state of maximal aggregated execution flow V/TV/T’’, that corresponds to eigenvalue problem

|V|ψ[i]=λ[i]|T|ψ[i]\displaystyle\Ket{V\middle|\psi^{[i]}}=\lambda^{[i]}\Ket{T\middle|\psi^{[i]}} (47)

and using this V/TV/T maximal λ\lambda in boundary condition (19). A remarkable feature of this state |ψ[VT]\Ket{\psi^{[VT]}} is that in it dV/dT=V/T=λ[V/T]dV/dT=V/T=\lambda^{[V/T]}:

ψ[VT]missing|I|ψ[VT]ψ[VT]|ψ[VT]\displaystyle\frac{\Braket{\psi^{[VT]}\left missing}{I}{\psi^{[VT]}}}{\Braket{\psi^{[VT]}}{\psi^{[VT]}}} =ψ[VT]missing|V|ψ[VT]ψ[VT]missing|T|ψ[VT]=SpurI|ρJVTSpurρJVT=λ[V/T]λ[IH]\displaystyle=\frac{\Braket{\psi^{[VT]}\left missing}{V}{\psi^{[VT]}}}{\Braket{\psi^{[VT]}\left missing}{T}{\psi^{[VT]}}}=\frac{\mathrm{Spur}\left\|I\middle|\rho_{JVT}\right\|}{\mathrm{Spur}\left\|\rho_{JVT}\right\|}=\lambda^{[V/T]}\leq\lambda^{[IH]} (48)

what allows us to consider ‘‘double integrated’’ type of the density matrix we considered in Appendix F. In this |ψ[VT]\Ket{\psi^{[VT]}} state we typically have SpurIdpdt|ρJVTSpurdpIdt|ρJVT\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JVT}\right\|\approx\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JVT}\right\|, no ‘‘advancing’’ information we managed to obtain so far.

When asked about ‘‘direct application’’ of the solution presented in Fig. 5, an ‘‘advancing’’ indicator, to practical trading – it is not ready for two reasons:

  • The result strongly depends on approximation used for Idpdt\left\|I\frac{dp}{dt}\right\|, see Appendix A.

  • In Fig. 5 we have shown only the interval, where predicted future price is substantially different from last price. For large intervals of this trading session the difference between predicted and last price is very small, this means the relation SpurpdIdt|ρJIH=SpurIdpdt|ρJIH\mathrm{Spur}\left\|p\frac{dI}{dt}\middle|\rho_{JIH}\right\|=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JIH}\right\| holds almost exactly in |ψ[IH]\Ket{\psi^{[IH]}} state.

This makes us to conclude that execution flow driven dynamics of Section IV.2, while is a very promising one and is the only one we obtained, that can possibly provide an ‘‘advancing’’ indicator – it still requires more work for practical applications. These two directions are the most promising:

  1. 1.

    Select other eigenvalue problem (6) that will be determining the time scale. The best what we found is (10) with the state |ψ[IH]\Ket{\psi^{[IH]}} of maximal eigenvalue λ[IH]\lambda^{[IH]}. All other tried have been worse.

  2. 2.

    Select directional indicator. The best indicator we found so far is the (35).

VI On Practical Source Of Directional Information

Whereas our attempts above to obtain an ‘‘advancing’’ indicator were promising but not ready yet for practical trading, a lagging directional indicator with automatic time–scale selection of Section IV.1, is ready and can be applied to trading. This indicator (29) is the P[IH]P^{[IH]} price plus trend-following factor that is proportional to dp/dtdp/dt in |ψ[IH]\Ket{\psi^{[IH]}} state:

PEQ\displaystyle P^{EQ} =P[IH]1λ[IH]ψ[IH]missing|(VVlast)dpdt|ψ[IH]\displaystyle=P^{[IH]}-\frac{1}{\lambda^{[IH]}}\Braket{\psi^{[IH]}\left missing}{(V-V^{last})\frac{dp}{dt}}{\psi^{[IH]}} (49)

The formula automatically selects the time scale from the interval [τ/n:τ][\tau/n:\tau] (in Shifted Legendre basis) or from the interval [τ:nτ][\tau:n\tau] (in Laguerre basis). The difference between last price and PEQP^{EQ} determines the trend. Trend ‘‘switch’’ occurs instantly as a ‘‘switch’’ in |ψ[IH]\Ket{\psi^{[IH]}} of (10) eigenproblem. In Fig. 8 below the (49) is presented in higher resolution than in Fig. 4 above. The situation when PEQP^{EQ} is close to PlastP^{last} corresponds to no information about the future (23) situation. Typically all the directional signals should be ignored (23) when

ψ0|ψ[IH]2\displaystyle\Braket{\psi_{0}}{\psi^{[IH]}}^{2} 0.1\displaystyle\gtrsim 0.1 (50)

as this corresponds to little information about the future available.

VII A Brief Description Of The Algorithm

Whereas a theory presented is this work may look rather complicated, it’s computer implementation is very straightforward. It is way simpler than multiple systems of other people the authors have seen in diversity \stackrel{{\scriptstyle\cdot\cdot}}{{\frown}}. Technically an implementation of the theory requires an integration to calculate the moments from timeserie sample (66), polynomials multiplication (53) to calculate the matrices from moments, and solving an eigenproblem (10) for time scale. There is no magic, simple and precise description of an algorithm implementing the theory is this: On each (Time, Execution Price, Shares Traded) tick coming do the following:

  1. 1.

    Have an integrator that on each tick coming recurrently updates an internal states to calculate the moments: Qm\Braket{Q_{m}}, QmI\Braket{Q_{m}I}, QmpI\Braket{Q_{m}pI}, and QmVdpdt\Braket{Q_{m}V\frac{dp}{dt}}; see \seqsplitcom/polytechnik/freemoney/CommonlyUsedMoments.java.

  2. 2.

    Using these moments construct the matrices Qj|Qk\Braket{Q_{j}}{Q_{k}}, Qj|I|Qk\Braket{Q_{j}}{I}{Q_{k}}, Qj|pI|Qk\Braket{Q_{j}}{pI}{Q_{k}}, and QmddtVdpdt\Braket{Q_{m}\frac{d}{dt}V\frac{dp}{dt}} by applying multiplication operator (53), then solve generalized eigenvalue problem (11) to find the state of maximal execution flow |ψ[IH]\Ket{\psi^{[IH]}} determining the time scale, see \seqsplitcom/polytechnik/freemoney/FreeMoneyForAll.java.

  3. 3.

    Construct a polynomial corresponding to the state ‘‘since |ψ[IH]\Ket{\psi^{[IH]}} till now’’ (60), then obtain corresponding density matrix ρJ(ψ[IH]2)\|\rho_{J\left({\psi^{[IH]}}^{2}\right)}\| using Theorem 3 of [7], see \seqsplitcom/polytechnik/freemoney/IandDM.java.

  4. 4.

    Obtain (49) price as directional indicator. Use (50) as non-applicability criterion. See \seqsplitcom/polytechnik/freemoney/PFuture.java:PEQV_from_M.

  5. 5.

    Optionally. Try (36), but this calculation requires an approximation for Idpdt\left\|I\frac{dp}{dt}\right\| and is much more sensitive to time-scale selection than (49). The result for ‘‘advancing’’ directional indicator (36) is not always satisfactory.

See Appendix C below with a description of the software implementing the algorithm. This software reads a sequence of (Time, Execution Price, Shares Traded) ticks (line after line, one tick per line), and for every tick read prints the results.

VIII Conclusion

An approach to obtain directional information from a sequence of past transactions with an automatic time–scale selection from execution flow I=dV/dtI=dV/dt is presented. Whereas a regular moving average has a built-in fixed time scale, the approach of this paper uses the state of maximal execution flow (10) to automatically determine the one. Contrary to regular moving average the developed approach has n1n-1 internal degrees of freedom to adjust averaging weight according to spikes in execution flow I=dV/dtI=dV/dt. These internal degrees of freedom allow to obtain an immediate ‘‘switch’’, what is not possible in regular moving average that always has a τ\tau-proportional time delay, lagging indicator. For a problem of dimension nn in Shifted Legendre basis the system automatically selects the time scale from the interval [τ/n:τ][\tau/n:\tau], and in Laguerre basis from the interval [τ:nτ][\tau:n\tau]. Among unsolved problem we would note a selection of optimal Idpdt\left\|I\frac{dp}{dt}\right\| interpolation to obtain an advancing price from (35), see Appendix A, and studying a possibility to ‘‘split’’ some average value based on some other operator spectrum, see Appendix E, Eq. (95). The software implementing the theory is available from the authors. Among directly applicable to trading results we would note the price (49) that includes both ‘‘switching’’ and ‘‘tending’’ contributions, see Fig. 8.

A generalization of the developed theory to a multi asset universe creates a number of new opportunities. Now from a sequence of past transactions l=1Ml=1\dots M for NaN_{a} financial instruments:

(tl,pl(a),dVl(a))\displaystyle\left(t_{l},p^{(a)}_{l},dV^{(a)}_{l}\right) a=1Na\displaystyle a=1\dots N_{a} (51)

one can construct NaN_{a} execution flow operators Qjmissing|I(a)|Qk\Braket{Q_{j}\left missing}{I^{(a)}}{Q_{k}}, each one with it’s own state |ψ[IH](a)\Ket{{\psi^{[IH]}}^{(a)}} of maximal I(a)I^{(a)} and corresponding to it ρJIH(a)\rho_{JIH}^{(a)}. In addition to this an operator of capital-flow index

I~\displaystyle\widetilde{I} =a=1Nap(a)dV(a)dt=a=1Nap(a)I(a)\displaystyle=\sum\limits_{a=1}^{N_{a}}p^{(a)}\frac{dV^{(a)}}{dt}=\sum\limits_{a=1}^{N_{a}}p^{(a)}I^{(a)} (52)

can be constructed to determine market overall activity; it also has it’s own state |ψ[IH]I~\Ket{{\psi^{[IH]}}^{\widetilde{I}}} of maximal I~\widetilde{I}444 A one of self-evident trading strategies: when current value of I~\widetilde{I} is large ψ0|ψ[IH]I~20.8\Braket{\psi_{0}}{{\psi^{[IH]}}^{\widetilde{I}}}^{2}\gtrsim 0.8 select the assets aa with currently low execution flow ψ0|ψ[IL](a)20.5\Braket{\psi_{0}}{{\psi^{[IL]}}^{(a)}}^{2}\gtrsim 0.5 as “lagging” and soon to follow in the direction of the market. . Critically important that all these |ψ\Ket{\psi} are in the same basis (the one of (56)) and their scalar products ψ|ϕ\Braket{\psi}{\phi} can be readily calculated. Technically this means we can independently use NaN_{a} integrators \seqsplitcom/polytechnik/freemoney/CommonlyUsedMoments.java, where each one calculates the moments Qmf(a)\Braket{Q_{m}f^{(a)}} of it’s own single asset a=1Naa=1\dots N_{a}, and then, from here, all the cross-asset characteristics can be calculated via projections! For example: how similar is the state of high execution flow of asset aa and the one of asset bb ?— it is just a regular scalar product of two wavefunctions ψ[IH](a)|ψ[IH](b)2\Braket{{\psi^{[IH]}}^{(a)}}{{\psi^{[IH]}}^{(b)}}^{2}; ‘‘correlated’’ assets are not the assets which prices ‘‘go together’’ but the assets with simultaneous spikes in execution flow. In addition to simultaneously criterion (projection) a criterion for ‘‘which one came earlier: a spike in I(a)\|I^{(a)}\| or a spike in I(b)\|I^{(b)}\|’’ can be written in a similar way: ψ[IH](a)|(tnowt)I(a)|ψ[IH](a)ψ[IH](a)|I(a)|ψ[IH](a)ψ[IH](b)|(tnowt)I(b)|ψ[IH](b)ψ[IH](b)|I(b)|ψ[IH](b)\frac{\Braket{{\psi^{[IH]}}^{(a)}}{(t_{now}-t)I^{(a)}}{{\psi^{[IH]}}^{(a)}}}{\Braket{{\psi^{[IH]}}^{(a)}}{I^{(a)}}{{\psi^{[IH]}}^{(a)}}}-\frac{\Braket{{\psi^{[IH]}}^{(b)}}{(t_{now}-t)I^{(b)}}{{\psi^{[IH]}}^{(b)}}}{\Braket{{\psi^{[IH]}}^{(b)}}{I^{(b)}}{{\psi^{[IH]}}^{(b)}}} obtained from directly sampled moments Qm(tnowt)I(a)\Braket{Q_{m}(t_{now}-t)I^{(a)}} and Qm(tnowt)I(b)\Braket{Q_{m}(t_{now}-t)I^{(b)}}, or as SpurρJIH(a)SpurρJIH(b)\mathrm{Spur}\|\rho^{(a)}_{JIH}\|-\mathrm{Spur}\|\rho^{(b)}_{JIH}\| what does not require other moments. There are several alternative forms of ‘‘distance’’ to determine which |ψ\Ket{\psi} happened earlier, see [1], ‘‘Appendix A: Time–Distance Between |ψ\Ket{\psi} States’’.

In a multi asset univers complexity of calculations growths linearly with NaN_{a}, hence the value of NaN_{a} can be very high even for realtime processing. Moreover, as every integrator \seqsplitcom/polytechnik/freemoney/CommonlyUsedMoments.java works independently, the problem can be easily parallelized to run each integrator on a separate core. Then all the cross-asset characteristics can be obtained from individual asset data (the moments from \seqsplitcom/polytechnik/freemoney/CommonlyUsedMoments.java instance) with standard linear algebra operations such as projection (scalar product), taking the difference between two SpurρJIH\mathrm{Spur}\|\rho_{JIH}\| to determine the distance, or considering some other operator (e.g. capital-flow index (52)) in a state like |ψ\Ket{\psi}, ρJ(ψ2)\|\rho_{J(\psi^{2})}\|, or ρJ(J(ψ2))\|\rho_{J(J(\psi^{2}))}\|.

We see an application of this paper theory to multi asset universe as the most promising direction of future research. The simplest, but really good, indicator is the Plast(a)PEQ(a){P^{last}}^{(a)}-{P^{EQ}}^{(a)} indicator (49) calculated for each a=1Naa=1\dots N_{a} asset then all summed with the λ[IH](a){\lambda^{[IH]}}^{(a)} weights for the terms in the sum to have the dimension of capital flow.

Acknowledgements.
This research was supported by Autretech group[9], www.атретек.рф. We thank our colleagues from Autretech R&D department who provided insight and expertise that greatly assisted the research. Our grateful thanks are also extended to Mr. Gennady Belov for his methodological support in doing the data analysis.

Appendix A On Calculation of QmIdpdt\Braket{Q_{m}I\frac{dp}{dt}} moments from QmpI\Braket{Q_{m}pI} sampled moments.

A theory developed in this paper works primarily with an observable ff and corresponding operator (matrix) Qj|f|Qk\Braket{Q_{j}}{f}{Q_{k}}, j,k=0n1j,k=0\dots n-1 that is obtained by applying multiplication operator:

QjQk\displaystyle Q_{j}Q_{k} =m=0j+kcmjkQm\displaystyle=\sum_{m=0}^{j+k}c_{m}^{jk}Q_{m} (53)

to sampled moments Qmf\Braket{Q_{m}f}, m=02n2m=0\dots 2n-2. The moments are defined with Qm(x)Q_{m}(x) being a polynomial of order mm and integration measure ω(t)dt\omega(t)\,dt having the support t[tnow]t\in[-\infty\dots t_{now}]:

Qmf\displaystyle\Braket{Q_{m}f} =tnow𝑑tω(t)Qm(x(t))f(t)\displaystyle=\int\limits_{-\infty}^{t_{now}}dt\,\omega(t)Q_{m}(x(t))f(t) (54)

In this paper we use: ω(t)\omega(t) is decaying exponent and x(t)x(t) is either linear or exponential function on time:

ω(t)\displaystyle\omega(t) =exp((tnowt)/τ)\displaystyle=\exp\left(-(t_{now}-t)/\tau\right) (55)
x(t)\displaystyle x(t) ={(ttnow)/τLaguerre basisexp((tnowt)/τ)shifted Legendre basis\displaystyle=\begin{cases}(t-t_{now})/\tau&\text{Laguerre basis}\\ \exp\left(-(t_{now}-t)/\tau\right)&\text{shifted Legendre basis}\end{cases} (56)

These two bases correspond to a more general (also analytically approachable) form x(t)=exp((tnowt)/τ)x(t)=\exp\left(-(t_{now}-t)/\tau^{*}\right), where ω(t)\omega(t) and x(t)x(t) are both exponential functions on tt but with different time scales: τ\tau and τ\tau^{*}. Other forms can also be considered.

If we want to consider df/dtdf/dt moments, then put it to (54) and do an integration by parts:

Qmdfdt\displaystyle\Braket{Q_{m}\frac{df}{dt}} =f(tnow)ω(x0)Qm(x0)tnow𝑑tf(t)ddtω(x(t))Qm(x(t))\displaystyle=f(t_{now})\omega(x_{0})Q_{m}(x_{0})-\int\limits_{-\infty}^{t_{now}}dt\,f(t)\frac{d}{dt}\omega(x(t))Q_{m}(x(t)) (57)

if ddtω(x(t))Qm(x(t))\frac{d}{dt}\omega(x(t))Q_{m}(x(t)) is equal to the same weight multiplied by a polynomial: ω(x)P(x)\omega(x)P(x) then the moments of df/dtdf/dt can be obtained from the moments of ff according to (57). The key element is an existence of ED()\mathrm{ED}(\cdot), a polynomial–to–polynomial mapping function (it is obtained as a derivative of a polynomial multiplied by the weight):

ddtω(x(t))ψ(x(t))φ(x(t))\displaystyle\frac{d}{dt}\omega(x(t))\psi(x(t))\varphi(x(t)) =ω(x)[ED(ψ)φ+ψED(φ)]\displaystyle=\omega(x)\left[\mathrm{ED}(\psi)\varphi+\psi\mathrm{ED}(\varphi)\right] (58)
ED(ψ(x))\displaystyle\mathrm{ED}(\psi(x)) ={dψ(x)dx+12ψ(x)Laguerre basisxdψ(x)dx+12ψ(x)shifted Legendre basis\displaystyle=\begin{cases}\displaystyle\frac{d\psi(x)}{dx}+\frac{1}{2}\psi(x)&\text{Laguerre basis}\\[10.00002pt] \displaystyle x\frac{d\psi(x)}{dx}+\frac{1}{2}\psi(x)&\text{shifted Legendre basis}\end{cases} (59)

where the time-derivative of a polynomial multiplied by a weight is represented by the same weight multiplied by other polynomial. The (57) corresponds to ψ=1\psi=1 and φ=Qm\varphi=Q_{m}.

For the two bases we consider in this paper it is also possible to obtain Qmf\Braket{Q_{m}f} moments from Qmdfdt\Braket{Q_{m}\frac{df}{dt}} moments using integration by parts, see [2], section ‘‘Basic Mathematics’’, about J()J(\cdot) polynomial-to-polynomial mapping such that for an arbitrary polynomial P(x)P(x):

tP(x(t))ω(t)𝑑t=ω(t)J(P)\displaystyle\int\limits_{-\infty}^{t}P(x(t^{\prime}))\omega(t^{\prime})dt^{\prime}=\omega(t)J(P) (60)

For the bases we use such a polynomial-to-polynomial transform exists:

J(P)\displaystyle J(P) ={1exp(x)xP(x)exp(x)𝑑xLaguerre basis1x0xP(x)𝑑xshifted Legendre basis\displaystyle=\begin{cases}{\displaystyle\frac{1}{\exp(x)}\int\limits_{-\infty}^{x}P(x^{\prime})\exp(x^{\prime})dx^{\prime}}&\text{Laguerre basis}\\ {\displaystyle\frac{1}{x}\int\limits_{0}^{x}P(x^{\prime})dx^{\prime}}&\text{shifted Legendre basis}\end{cases} (61)

A remarkable feature of this transform is that since (f(tnow)f)P=J(P)dfdt\Braket{(f(t_{now})-f)P}=\Braket{J(P)\frac{df}{dt}} and J(P)J(P) is also a polynomial, thus an average with it can be converted to a density matrix average, any average Pf\Braket{Pf} can be represented as the spur from a product of operator df/dt\|df/dt\| and a density matrix ρ\|\rho\|:

Pf\displaystyle\Braket{Pf} =Spurρ|dfdt\displaystyle=\mathrm{Spur}\left\|\rho\middle|\frac{df}{dt}\right\| (62)

This way any average of an observable ff can be calculated as operator df/dt\|df/dt\| averaged in some mixed state ρ\rho obtained from the polynomial P(x)P(x). Note, that J()J(\cdot) transform can be applied in chain:

tnowdfdtJ(P)ω(t)𝑑t\displaystyle\int\limits_{-\infty}^{t_{now}}\frac{df}{dt}J(P)\omega(t)dt =f|x0J(P)|x0ω(x0)tnowfP(x(t))ω(t)𝑑t\displaystyle=f\Big{|}_{x_{0}}J(P)\Big{|}_{x_{0}}\omega(x_{0})-\int\limits_{-\infty}^{t_{now}}fP(x(t))\omega(t)dt (63)
tnowd2fdt2J(J(P))ω(t)𝑑t\displaystyle\int\limits_{-\infty}^{t_{now}}\frac{d^{2}f}{dt^{2}}J(J(P))\omega(t)dt =dfdt|x0J(J(P))|x0ω(x0)f|x0J(P)|x0ω(x0)+tnowfP(x(t))ω(t)𝑑t\displaystyle=\frac{df}{dt}\Big{|}_{x_{0}}J(J(P))\Big{|}_{x_{0}}\omega(x_{0})-f\Big{|}_{x_{0}}J(P)\Big{|}_{x_{0}}\omega(x_{0})+\int\limits_{-\infty}^{t_{now}}fP(x(t))\omega(t)dt (64)

The moments of ff are usually obtained from direct sampling of all available observations l=1Ml=1\dots M in a timeserie:

Qmf\displaystyle\Braket{Q_{m}f} =l=1Mf(tl)Qm(x(tl))ω(tl)[tltl1]\displaystyle=\sum_{l=1}^{M}f(t_{l})Q_{m}(x(t_{l}))\omega(t_{l})\left[t_{l}-t_{l-1}\right] (65)

the moments of a derivative df/dtdf/dt can also be obtained from direct sampling:

Qmdfdt\displaystyle\Braket{Q_{m}\frac{df}{dt}} =l=1MQm(x(tl))ω(tl)[f(tl)f(tl1)]\displaystyle=\sum_{l=1}^{M}Q_{m}(x(t_{l}))\omega(t_{l})\left[f(t_{l})-f(t_{l-1})\right] (66)

See [1], section ‘‘Basis Selection’’, for one more basis: price basis: Qk(t)=pk(t)Q_{k}(t)=p^{k}(t). It has no ED()\mathrm{ED}(\cdot) and J()J(\cdot) operators available, but has similar sampling formula. Given a good choice of basis polynomials:

Qm(x)\displaystyle Q_{m}(x) ={Lm(x)Laguerre basisPm(2x1)shifted Legendre basis\displaystyle=\begin{cases}L_{m}(-x)&\text{Laguerre basis}\\ P_{m}(2x-1)&\text{shifted Legendre basis}\end{cases} (67)

one can calculate (with double precision arithmetic) the moments to a very high order m50m\lesssim 50 (limited by the divergence of cmjkc_{m}^{jk} multiplication coefficients (53)) in Laguerre basis, and m150m\lesssim 150 (limited by poorly conditioned matrices) in shifted Legendre basis; Chebyshev polynomials Tm(2x1)T_{m}(2x-1) also provide very stable calculations in shifted Legendre basis (Chebyshev polynomials have perfectly stable multiplication: all cmjk=0c_{m}^{jk}=0 except cjkjk=cj+kjk=0.5c_{j-k}^{jk}=c_{j+k}^{jk}=0.5, jkj\geq k). The result is invariant with respect to basis choice, Qm(x)=xmQ_{m}(x)=x^{m} and the ones from (67) give identical results, but numerical stability can be drastically different[10, 3].

Moments calculated from market data timeserie using Eqs. (65) and (66) are the cornerstone of our theory. The most important are the moments of execution flow I=dV/dtI=dV/dt, they are obtained from (66) by putting the volume as f=Vf=V, thus the moments QmI\Braket{Q_{m}I} are obtained from timeserie sample; the matrix Qj|I|Qk\Braket{Q_{j}}{I}{Q_{k}} is obtained from them using multiplication operator (53). The matrix Qj|Qk\Braket{Q_{j}}{Q_{k}} is known analytically. These two matrices are volume- and time- averaged products of two basis functions. A generalized eigenvalue problem (5) is then formulated:

|I|ψ[i]\displaystyle\Ket{I\middle|\psi^{[i]}} =λ[i]|ψ[i]\displaystyle=\lambda^{[i]}\Ket{\psi^{[i]}} (68)
k=0n1Qj|I|Qkαk[i]\displaystyle\sum\limits_{k=0}^{n-1}\Braket{Q_{j}}{I}{Q_{k}}\alpha^{[i]}_{k} =λ[i]k=0n1Qj|Qkαk[i]\displaystyle=\lambda^{[i]}\sum\limits_{k=0}^{n-1}\Braket{Q_{j}}{Q_{k}}\alpha^{[i]}_{k} (69)
ψ[i](x)\displaystyle\psi^{[i]}(x) =k=0n1αk[i]Qk(x)\displaystyle=\sum\limits_{k=0}^{n-1}\alpha^{[i]}_{k}Q_{k}(x) (70)

and solved. Whereas the calculation of the moments QmI\Braket{Q_{m}I}, QmpI\Braket{Q_{m}pI}, Qmdpdt\Braket{Q_{m}\frac{dp}{dt}}, QmVdpdt\Braket{Q_{m}V\frac{dp}{dt}}, Qmp\Braket{Q_{m}p} create no problem whatsoever, an attempt to go beyond them turned out to be problematic. For example any second order derivative (e.g. d2p/dt2d^{2}p/dt^{2}) cannot be obtained directly from sample (66) and, in the same time, has singularities when applying an integration by parts (57), not to mention difficulties to formulate a boundary condition at x=x0x=x_{0}.

However, some of these characteristics are of great interest. The most important one is QmIdpdt\Braket{Q_{m}I\frac{dp}{dt}}. The price p=dpdt𝑑t=𝑑pp=\int\frac{dp}{dt}dt=\int dp provides an information of current market state, but little one about possible trading opportunities. Assume at a given time interval dtdt we have some specific constant value of execution flow I=dV/dtI=dV/dt and some dp/dtdp/dt. During this dtdt interval dV=IdtdV=Idt shares were traded and the price change was dp=dpdtdtdp=\frac{dp}{dt}dt. How much money we potentially can make during this dtdt? Buy at the beginning of tt: dVdV shares at pp. Sell at the end of t+dtt+dt: dVdV shares at p+dpp+dp. Total potential P&L (assuming we can perfectly frontrun the market555 During every dtdt we hold dVdV shares, i.e. we always hold a position SS equals to execution flow I=dV/dtI=dV/dt, the P&L=S(t)𝑑p\mathrm{P\&L}=\int S(t)dp, see [3], Section “P&L operator and trading strategy”.) is then dP&L=Idpd\mathrm{P\&L}=Idp. The P&L=I𝑑p\mathrm{P\&L}=\int Idp tells us how much money can be potentially made (or lost) on market movements taking into account traded volume capacity. This is the same sum of price changes as for regular price p=𝑑pp=\int dp, but not all dpdp are created equal. If dpdp occurred on a large execution flow – it contributes more, if on a small – it contributes less666 The concept of Idpdt=dVdtdpdtI\frac{dp}{dt}=\frac{dV}{dt}\frac{dp}{dt} is very different from commonly studied market impact concept that is a price sensitivity to volume traded: dp/dVdp/dV. This creates a different way to study opportunities of market movements, see Fig. 6.

Refer to caption
Figure 6: A schematic example of price (the sum of price elementary changes dpdp) vs. possible P&L (the sum of price changes multiplied by execution flow IdpIdp). The second curve tells how much money one can be potentially made (or lost) on market movements taking into account traded volume capacity.

The value of QmIdpdt\Braket{Q_{m}I\frac{dp}{dt}} cannot be calculated directly as well as it cannot be calculated using integration by parts (57) in a general basis. However, if we change the basis and vary basis functions in the basis of (68) eigenproblem |I|ψ[i]=λ[i]|ψ[i]\Ket{I\middle|\psi^{[i]}}=\lambda^{[i]}\Ket{\psi^{[i]}} an approximate solution can be obtained. Consider the basis |ψ[i]\Ket{\psi^{[i]}}, it is orthogonal (8), (9) as:

δjk=ψ[j]|ψ[k]\displaystyle\delta_{jk}=\Braket{\psi^{[j]}}{\psi^{[k]}} (71a)
λ[j]δjk=ψ[j]|I|ψ[k]\displaystyle\lambda^{[j]}\delta_{jk}=\Braket{\psi^{[j]}}{I}{\psi^{[k]}} (71b)

what gives φ|I|ψ[i]=λ[i]φ|ψ[i]\Braket{\varphi}{I}{\psi^{[i]}}=\lambda^{[i]}\Braket{\varphi}{\psi^{[i]}} for an arbitrary |φ\Ket{\varphi}. Then taking into account (57) and (58) put f=pIf=pI and Plast=p(tnow)P^{last}=p(t_{now}). Obtain for matrix elements:

ψ[j]missing|dpIdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{\frac{dpI}{dt}}{\psi^{[k]}} =PlastI(tnow)ω(x0)ψ[j](x0)ψ[k](x0)\displaystyle=P^{last}I(t_{now})\omega(x_{0})\psi^{[j]}(x_{0})\psi^{[k]}(x_{0})
ED(ψ[j])|pI|ψ[k]ψ[j]|pI|ED(ψ[k])\displaystyle-\Braket{\mathrm{ED}(\psi^{[j]})}{pI}{\psi^{[k]}}-\Braket{\psi^{[j]}}{pI}{\mathrm{ED}(\psi^{[k]})} (72)

These are matrix elements of ψ[j]missing|dpIdt|ψ[k]\Braket{\psi^{[j]}\left missing}{\frac{dpI}{dt}}{\psi^{[k]}}. We are going to modify them to obtain sought matrix elements ψ[j]missing|Idpdt|ψ[k]\Braket{\psi^{[j]}\left missing}{I\frac{dp}{dt}}{\psi^{[k]}}, that should be zero when p=constp=const. For this reason the average in (72) should be reduced to an integral of an exact differential to be canceled with out of integral term. Taking into account (68) and (71) one can see that

ψ[j]missing|Idpdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{I\frac{dp}{dt}}{\psi^{[k]}} =Plastλ[j]λ[k]ω(x0)ψ[j](x0)ψ[k](x0)\displaystyle=P^{last}\sqrt{\lambda^{[j]}\lambda^{[k]}}\omega(x_{0})\psi^{[j]}(x_{0})\psi^{[k]}(x_{0})
λ[j]λ[k]ED(ψ[j])|pI|ψ[k]λ[k]λ[j]ψ[j]|pI|ED(ψ[k])\displaystyle-\sqrt{\frac{\lambda^{[j]}}{\lambda^{[k]}}}\Braket{\mathrm{ED}(\psi^{[j]})}{pI}{\psi^{[k]}}-\sqrt{\frac{\lambda^{[k]}}{\lambda^{[j]}}}\Braket{\psi^{[j]}}{pI}{\mathrm{ED}(\psi^{[k]})} (73)

satisfies exact differential condition. Then practically applicable expression is:

ψ[j]missing|Idpdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{I\frac{dp}{dt}}{\psi^{[k]}} =\displaystyle= λ[j]λ[k]ED(ψ[j])|(Plastp)I|ψ[k]+λ[k]λ[j]ψ[j]|(Plastp)I|ED(ψ[k])\displaystyle\sqrt{\frac{\lambda^{[j]}}{\lambda^{[k]}}}\Braket{\mathrm{ED}(\psi^{[j]})}{(P^{last}-p)I}{\psi^{[k]}}+\sqrt{\frac{\lambda^{[k]}}{\lambda^{[j]}}}\Braket{\psi^{[j]}}{(P^{last}-p)I}{\mathrm{ED}(\psi^{[k]})} (74)

This is an expression of operator Idpdt\left\|I\frac{dp}{dt}\right\| in the basis of (68). The reason why we were able to obtain this explicit expression is that we managed to combine differentiation of a product (58) with eigenvalues problem (68) to write, when p=constp=const, each matrix element as an exact differential. This approximation can also be viewed as operators multiplication factoring:

φ|pI|ψ[i]\displaystyle\Braket{\varphi}{pI}{\psi^{[i]}} k=0n1φ|p|ψ[k]ψ[k]|I|ψ[i]=φ|p|ψ[i]λ[i]\displaystyle\approx\sum\limits_{k=0}^{n-1}\Braket{\varphi}{p}{\psi^{[k]}}\Braket{\psi^{[k]}}{I}{\psi^{[i]}}=\Braket{\varphi}{p}{\psi^{[i]}}\lambda^{[i]} (75)

what gives

DPjk=1λ[k]ED(ψ[j])|(Plastp)I|ψ[k]+1λ[j]ψ[j]|(Plastp)I|ED(ψ[k])\displaystyle\mathrm{DP}_{jk}=\frac{1}{\lambda^{[k]}}\Braket{\mathrm{ED}(\psi^{[j]})}{(P^{last}-p)I}{\psi^{[k]}}+\frac{1}{\lambda^{[j]}}\Braket{\psi^{[j]}}{(P^{last}-p)I}{\mathrm{ED}(\psi^{[k]})} (76)

and two approximations for Idpdt\|I\frac{dp}{dt}\|:

ψ[j]missing|Idpdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{I\frac{dp}{dt}}{\psi^{[k]}} λ[j]λ[k]DPjk=I1/2DPI1/2\displaystyle\approx\sqrt{\lambda^{[j]}\lambda^{[k]}}\mathrm{DP}_{jk}=\|I^{1/2}\|\cdot\|\mathrm{DP}\|\cdot\|I^{1/2}\| (77)
ψ[j]missing|Idpdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{I\frac{dp}{dt}}{\psi^{[k]}} DPI\displaystyle\approx\|\mathrm{DP}\|\cdot\|I\| (78)

Using an approximation IβdpdtIβ2|dpdt|Iβ2\|I^{\beta}\frac{dp}{dt}\|\approx\|I^{\frac{\beta}{2}}|\frac{dp}{dt}|I^{\frac{\beta}{2}}\| the (77) result can be generalized to a II–power β\beta, however this result is not very accurate for β\beta other than 0 or 11, since it is obtained from the QmpI\Braket{Q_{m}pI} moments:

ψ[j]missing|Iβdpdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{I^{\beta}\frac{dp}{dt}}{\psi^{[k]}} (λ[j]λ[k])β2DPjk=Iβ/2DPIβ/2\displaystyle\approx\left(\lambda^{[j]}\lambda^{[k]}\right)^{\frac{\beta}{2}}\mathrm{DP}_{jk}=\|I^{\beta/2}\|\cdot\|\mathrm{DP}\|\cdot\|I^{\beta/2}\| (79)

This method of calculation is implemented in \seqsplitcom/polytechnik/freemoney/MatricesFromPI.java:getbQQIpowdpdtFromQQpi_withLastP; for DPjk\mathrm{DP}_{jk} see e.g. \seqsplitcom/polytechnik/freemoney/PFuture.java:bQQ_DP. A matrix obtained in |ψ[i]\Ket{\psi^{[i]}} basis can be converted to QjQ_{j} basis in a regular way:

Qj|f|Qk\displaystyle\Braket{Q_{j}}{f}{Q_{k}} =l,i,s,m=0n1Gjlαl[i]ψ[i]|f|ψ[s]αm[s]Gmk\displaystyle=\sum\limits_{l,i,s,m=0}^{n-1}G_{jl}\alpha^{[i]}_{l}\Braket{\psi^{[i]}}{f}{\psi^{[s]}}\alpha^{[s]}_{m}G_{mk} (80)
ψ[j]|f|ψ[k]\displaystyle\Braket{\psi^{[j]}}{f}{\psi^{[k]}} =s,m=0n1αs[j]Qs|f|Qmαm[k]\displaystyle=\sum\limits_{s,m=0}^{n-1}\alpha^{[j]}_{s}\Braket{Q_{s}}{f}{Q_{m}}\alpha^{[k]}_{m} (81)

where Gram matrix Gjk=Qj|QkG_{jk}=\Braket{Q_{j}}{Q_{k}}, and αk[i]\alpha^{[i]}_{k} are (68) eigenvectors in QjQ_{j} basis, Eq. (70), see \seqsplitcom/polytechnik/utils/EVXData.java. 777 A question arise whether obtained matrix Qjmissing|Iβdpdt|Qk\Braket{Q_{j}\left missing}{I^{\beta}\frac{dp}{dt}}{Q_{k}}, j,k=0n1j,k=0\dots n-1 corresponds to a measure or not: Whether it can be obtained from some QmIβdpdt\Braket{Q_{m}I^{\beta}\frac{dp}{dt}} moments, m=02n2m=0\dots 2n-2 by applying multiplication operator (53)? We have an algorithm to establish this fact, see Theorem 3 of [7]. Numerical experiments show that this is almost the case. A mismatch may be caused either by some numerical instability or by some degeneracy in the problem. A numerical instability is very unlikely because (62) holds exactly for both: 1). original Qjmissing|Iβdpdt|Qk\Braket{Q_{j}\left missing}{I^{\beta}\frac{dp}{dt}}{Q_{k}} matrix (77) and 2). the matrix obtained by applying multiplication operator (53) to the moments QmIβdpdt\Braket{Q_{m}I^{\beta}\frac{dp}{dt}} obtained from Theorem 3 of [7] applied to the original matrix (77). A mismatch between these two matrices is observed starting with n3n\geq 3; the difference is very small but clearly established numerically. This degeneracy (if exists) does not create any problem in calculation of the value of any observable as the density matrix used has the form: a state since |ψ[IH]\Ket{\psi^{[IH]}} spike. The expression (77) is an approximation, similar (and slightly better) approximation is a non–Hermitian matrix (78) as it has a single approximate product compared with two in (77).

Different approximations can be obtained using a general form of (75):

fg\displaystyle\left\|fg\right\| fg\displaystyle\approx\left\|f\right\|\cdot\left\|g\right\| (82)

corresponding to an approximation δ(xy)j,k=0n1Qj(x)Gjk1Qk(y)\delta(x-y)\approx\sum_{j,k=0}^{n-1}Q_{j}(x)G^{-1}_{jk}Q_{k}(y). We can obtain a result applying it to other moments available directly from sample: QmVdpdt\Braket{Q_{m}V\frac{dp}{dt}}, Qmdpdt\Braket{Q_{m}\frac{dp}{dt}} along with their derivatives QmdIdt\Braket{Q_{m}\frac{dI}{dt}}, QmdpIdt\Braket{Q_{m}\frac{dpI}{dt}}, QmddtVdpdt\Braket{Q_{m}\frac{d}{dt}V\frac{dp}{dt}}, and Qmd2pdt2\Braket{Q_{m}\frac{d^{2}p}{dt^{2}}} obtained from integration by parts formula (57) with boundary conditions: 1). impact from the future (19) and 2). zero VV at tnowt_{now}; traded volume VV is measured relatively tnowt_{now}, it is negative for past observations. A few useful approximations of operator Idpdt\left\|I\frac{dp}{dt}\right\|:

dpdtI\displaystyle\left\|\frac{dp}{dt}\right\|\cdot\left\|I\right\| (83)
dpIdtpdIdt\displaystyle\left\|\frac{dpI}{dt}\right\|-\left\|p\right\|\cdot\left\|\frac{dI}{dt}\right\| (84)
ddtVdpdtVd2pdt2\displaystyle\left\|\frac{d}{dt}V\frac{dp}{dt}\right\|-\left\|V\right\|\cdot\left\|\frac{d^{2}p}{dt^{2}}\right\| (85)

Despite these operators being non-Hermitian this creates no problem as they are used only in calculation of Spur\mathrm{Spur} with Hermitian density matrix such as in (45). The approximation (83) uses directly sampled moments Qmdpdt\Braket{Q_{m}\frac{dp}{dt}}, it is a product of two separately sampled operators with spikes, nevertheless it gives very similar to (78) results, without spurious artifacts; sometimes, however, there is a difficulty to combine it with dpIdt\|\frac{dpI}{dt}\| from (34), as in this case the moments from two different samplings are used together. The approximation (84) uses exact dpIdt\left\|\frac{dpI}{dt}\right\| and dIdt\left\|\frac{dI}{dt}\right\| operators matrix elements but the result is noisy. The (85) also uses exact values of ddtVdpdt\left\|\frac{d}{dt}V\frac{dp}{dt}\right\| and d2pdt2\left\|\frac{d^{2}p}{dt^{2}}\right\| (obtained from (57) with zero boundary condition due to VV factor) operators, but it is completely useless due to d2p/dt2d^{2}p/dt^{2} singularities; however without the last term the operator ddtVdpdt\left\|\frac{d}{dt}V\frac{dp}{dt}\right\| gives similar to dpIdt\left\|\frac{dpI}{dt}\right\| results. Two these operators should probably be considered together as corresponding to VdpVdp and pdVpdV. In sufficiently localizes states (e.g. |ψ[IH]\Ket{\psi^{[IH]}}) we typically have ψ[IH]|Vdpdt|ψ[IH]ψ[IH]|V|ψ[IH]ψ[IH]|dpdt|ψ[IH]\Braket{\psi^{[IH]}}{V\frac{dp}{dt}}{\psi^{[IH]}}\approx\Braket{\psi^{[IH]}}{V}{\psi^{[IH]}}\Braket{\psi^{[IH]}}{\frac{dp}{dt}}{\psi^{[IH]}}.

Another possible approach to obtain Idpdt\left\|I\frac{dp}{dt}\right\| that enters into (35) together with pdIdt\left\|p\frac{dI}{dt}\right\| is to consider the operator ddtpI\left\|\frac{d}{dt}\frac{p}{I}\right\|. As above, let us take (72) and, assuming that changes in pp are much smaller than changes in II, modify it to obtain ddtpI\left\|\frac{d}{dt}\frac{p}{I}\right\| matrix elements:

ψ[j]missing|ddtpI|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{\frac{d}{dt}\frac{p}{I}}{\psi^{[k]}} PlastI0Fω(x0)ψ[j](x0)ψ[k](x0)\displaystyle\approx\frac{P^{last}}{I_{0}^{F}}\omega(x_{0})\psi^{[j]}(x_{0})\psi^{[k]}(x_{0})
1λ[k]2ED(ψ[j])|pI|ψ[k]1λ[j]2ψ[j]|pI|ED(ψ[k])\displaystyle-\frac{1}{{\lambda^{[k]}}^{2}}\Braket{\mathrm{ED}(\psi^{[j]})}{pI}{\psi^{[k]}}-\frac{1}{{\lambda^{[j]}}^{2}}\Braket{\psi^{[j]}}{pI}{\mathrm{ED}(\psi^{[k]})} (86)

This expression satisfies limit case conditions. When p=constp=const the result exactly equals to ddt1I\left\|\frac{d}{dt}\frac{1}{I}\right\|, when I=constI=const the result exactly equals to dpdt\left\|\frac{dp}{dt}\right\|, and when pI=II\|pI\|=\|I\|\cdot\|I\| the result is a differential of a constant. With I0F=λ[IH]I_{0}^{F}=\lambda^{[IH]} as per (19), the (86) also satisfies Spurddt1I|ρJIH=0\mathrm{Spur}\left\|\frac{d}{dt}\frac{1}{I}\middle|\rho_{JIH}\right\|=0, the same as for SpurddtI|ρJIH=0\mathrm{Spur}\left\|\frac{d}{dt}I\middle|\rho_{JIH}\right\|=0. See \seqsplitcom/polytechnik/freemoney/MatricesFromPI.java:getbQQDtpDivIFromQQpi_withLastP for numerical implementation. This approximation ddtpI=Idp/dtpdI/dtI2\left\|\frac{d}{dt}\frac{p}{I}\right\|=\left\|\frac{Idp/dt-pdI/dt}{I^{2}}\right\| can be tried as a proxy to Idp/dtpdI/dt\left\|Idp/dt-pdI/dt\right\| in (35), but the result is very poor. The problem is that (86) has an extra common factor 1/I21/I^{2}. Similarly to (76) we can modify it by λ[j]λ[k]\lambda^{[j]}\lambda^{[k]} factor to remove I2I^{2} from the denominator:

ψ[j]|IdpdtpdIdt|ψ[k]\displaystyle\left\|\psi^{[j]}\middle|I\frac{dp}{dt}-p\frac{dI}{dt}\middle|\psi^{[k]}\right\| IddtpII\displaystyle\approx\left\|I\right\|\cdot\left\|\frac{d}{dt}\frac{p}{I}\right\|\cdot\left\|I\right\| (87)
ψ[j]missing|IdpdtpdIdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{I\frac{dp}{dt}-p\frac{dI}{dt}}{\psi^{[k]}} PlastI0Fω(x0)λ[j]λ[k]ψ[j](x0)ψ[k](x0)\displaystyle\approx\frac{P^{last}}{I_{0}^{F}}\omega(x_{0})\lambda^{[j]}\lambda^{[k]}\psi^{[j]}(x_{0})\psi^{[k]}(x_{0})
λ[j]λ[k]ED(ψ[j])|pI|ψ[k]λ[k]λ[j]ψ[j]|pI|ED(ψ[k])\displaystyle-\frac{\lambda^{[j]}}{\lambda^{[k]}}\Braket{\mathrm{ED}(\psi^{[j]})}{pI}{\psi^{[k]}}-\frac{\lambda^{[k]}}{\lambda^{[j]}}\Braket{\psi^{[j]}}{pI}{\mathrm{ED}(\psi^{[k]})} (88)

but this creates a problem that the condition SpurI|ddt1I|I|ρJIH=0\mathrm{Spur}\left\|I\middle|\frac{d}{dt}\frac{1}{I}\middle|I\middle|\rho_{JIH}\right\|=0 no longer holds, thus it cannot be applied in (35). One may try to adjust the value of I0FI_{0}^{F} in (88) to have this condition satisfied (put p=constp=const and take the Spur with ρJIH\|\rho_{JIH}\| of (88), let it equals to zero; the values of I0FI_{0}^{F} and ‘‘adjusted’’ I0FI_{0}^{F} are typically very close, adjusted value is often slightly larger). An important difference between Idp/dtpdI/dtI2\left\|\frac{Idp/dt-pdI/dt}{I^{2}}\right\| and Idp/dtpdI/dt\left\|Idp/dt-pdI/dt\right\| is that the first one is an exact differential (thus it’s Spur with ρJIH\rho_{JIH} can be reduced to sub-differential expression in |ψ[IH]\Ket{\psi^{[IH]}} state and the boundary term), and the second one is not, thus it cannot be reduced to some observable in |ψ[IH]\Ket{\psi^{[IH]}} state.

The (78), (83), and (88) are approximations that can be used for operator Idpdt\left\|I\frac{dp}{dt}\right\|, see \seqsplitcom/polytechnik/freemoney/PFuture.java for numerical implementation.

Appendix B On Surrogate Volume

Refer to caption
Figure 7: Averages: T[IH]T^{[IH]} (17) and regular moving average (18) (dark blue) TτT^{\tau} for τ=256\tau=256s; the values are multiplied by 10310^{-3} and shifted up to fit the chart). The time-to now calculations with real volume (green) and surrogate volume (light blue). Real volume VV and surrogate volume AA are also presented (scaled by 10610^{-6} and 51035\cdot 10^{-3} respectively) and shifted up to fit the chart). The AAPL stock on September, 20, 2012. The calculations in shifted Legendre basis with n=12n=12.

Another question of interest is whether all the developed theory and software of this work can be used without trading volume available. For a number of markets (such as: sovereign CDS, corporate fixed income, crypto exchanges, currency trading, etc.) it is quite common to have the price to be very accurate and available almost realtime, but the traded volume is either not available at all or provided incorrectly (sometimes intentionally incorrectly). In such a case there is an option to use the absolute value of price change as it were the volume; the calculations are the same – in (66) instead of df=VlVl1df=V_{l}-V_{l-1} one can use da=|plpl1|da=|p_{l}-p_{l-1}|. The only problem with this approach is that market events without price change would not be taken into account as for them da=0da=0, hence the results will be less accurate. However, our past experiments, see [3], Fig. 6, show that absolute value of price tick as a ‘‘poor man volume’’ often provides quite similar results. A feed of ‘‘all price ticks’’ can be used as a surrogate volume. For a liquid asset it typically takes a few minutes for da\sum da to exceed PlastP0=dpP^{last}-P^{0}=\sum dp in several orders of magnitude; for the entire trading session in Fig. 7 a typical maximal price change is about 55, but the sum of all absolute price changes is about 500500; total reported by NASDAQ ITCH [8] trading volume of this session is about 31063\cdot 10^{6} shares. As this dada sequence is all positive, we also tried it with the market impact dp/dVdp/dV concept (that completely failed with actual price change dpdp) in a hope that with this dada we may find an identifiable limit for dV/dadV/da. The result is also unsatisfactory: The dV/dadV/da is very similar to dV/dtdV/dt: it fluctuates in orders of magnitude and clearly has no stable limit at any time scale below 10 minutes (for US equity marker). However the da/dtda/dt is similar to dV/dtdV/dt and for liquid assets can be used as a ‘‘poor man volume’’ I=dadtI=\frac{da}{dt} with the matrices Qj|dadt|Qk\Braket{Q_{j}}{\frac{da}{dt}}{Q_{k}} and Qj|Qk\Braket{Q_{j}}{Q_{k}} in (69). See \seqsplitcom/polytechnik/freemoney/CommonlyUsedMoments.java:addObservationNoBasisShift for calculation of surrogate volume moments.

Refer to caption
Refer to caption
Figure 8: The equilibrium price PEQP^{EQ} from (29) for real volume PIEQP^{EQ}_{I} and surrogate volume PAEQP^{EQ}_{A}.

In Fig. 7 we present T[IH]T^{[IH]} (17) average (along with regular moving average TτT^{\tau}) calculated for regular volume dVdV and surrogate volume da=|plpl1|da=|p_{l}-p_{l-1}|. One can see similar behavior of localization ‘‘switches’’. However, surrogate volume states have some ‘‘switches’’ missed and overall picture is less detailed. In Fig. 8 the price PEQP^{EQ} from (29) is presented. One can see similar, but less detailed picture. This makes us to conclude that da=|plpl1|da=|p_{l}-p_{l-1}| is a ‘‘poor man volume’’.

Appendix C Software Usage Description

The software is written in java. As with [1] follow the steps:

  • Install java 19 or later.

  • Download from [11] NASDAQ ITCH data file \seqsplitS092012-v41.txt.gz, and the archive \seqsplitAMuseOfCashFlowAndLiquidityDeficit.zip with the source code. There is also an alternative location of these files.

  • Decompress and recompile the program:

    unzip AMuseOfCashFlowAndLiquidityDeficit.zip
    javac -g com/polytechnik/*/*java
    
  • Run the command to test the program

    java com/polytechnik/algorithms/TestCall_FreeMoneyForAll \
        --musein_file=dataexamples/aapl_old.csv.gz \
        --musein_cols=9:1:2:3 \
        --n=12 \
        --tau=256 \
        --measure=CommonlyUsedMomentsLegendreShifted \
        --museout_file=museout.dat
    

    Program parameters are:

    • --musein_file=aapl.csv : Input tab–separated file with (time, execution price, shares traded) triples timeserie. The file is possibly gzip-compressed.

    • --musein_cols=9:1:2:3 : Out of total 9 columns of dataexamples/aapl_old.csv.gz file, take column #1 as time (nanoseconds since midnight), #2 (execution price), and #3 (shares traded), column index is base 0.

    • --museout_file=museout.dat : Output file name is set to \seqsplitmuseout.dat.

    • --n=12 : Basis dimension. Typical values are: 2 (for testing a concept), or some value about [412][4\dots 12] for more advance use.

    • --tau=256 : Exponent time (in seconds) for the measure used.

    • --measure=CommonlyUsedMomentsLegendreShifted The measure. The values \seqsplitCommonlyUsedMomentsLaguerre,CommonlyUsedMomentsMonomials correspond to Laguerre measure and \seqsplitCommonlyUsedMomentsLegendreShifted corresponds to shifted Legendre measures. The results of \seqsplitCommonlyUsedMomentsMonomials (uses Qk(x)=xkQ_{k}(x)=x^{k}) should be identical to \seqsplitCommonlyUsedMomentsLaguerre (uses Qk(x)=Lk(x)Q_{k}(x)=L_{k}(-x)), as the measure is the same and all the calculations are Qk(x)Q_{k}(x)–basis invariant (but numerical stability is worse for \seqsplitCommonlyUsedMomentsMonomials).

  • The results are saved in output file museout.dat. Among the values of interest are the following:

    • pFV.pv_average Regular moving average PτP^{\tau} from (16).

    • pFV.Tv_average The moving average TτT^{\tau} from (18).

    • pFV.totalVolume Total volume traded up to current tick.

    • pFV.pv_M The value of P[IH]P^{[IH]} (15).

    • pFV.Tv_M The value of T[IH]T^{[IH]} (17), an indicator of localization of |ψ[IH]\Ket{\psi^{[IH]}}.

    • pFV.I.wH_squared The value of ψ0|ψ[IH]2\Braket{\psi_{0}}{\psi^{[IH]}}^{2} (50), an indicator of applicability of any prediction.

    • pFV.PEQV_from_M The value of predicted ‘‘lagging price’’ PEQP^{EQ} from (49) that includes both P[IH]P^{[IH]} and trending term, the best directional indicator we managed to obtain so far, see Fig. 8.

The output includes two versions: calculated with ‘‘actual’’ volume (prefixed with pFV.) and calculated with ‘‘surrogate volume’’ of Appendix B (prefixed with pFA.).

C.1 Software Code Structure

Provided software is located in several directories:

  • com/polytechnik/utils/ General basis utilities including my Radon-Nikodym approach [6] to machine learning.

  • com/polytechnik/lapack/ Ported to java LAPACK library.

  • com/polytechnik/lapack/ NASDAQ ITCH [8] parsing.

  • com/polytechnik/trading/ Both: ‘‘scaffolding’’ for new ideas and a ‘‘graveyard’’ for old ones. Also contains unit tests. One can run all unit tests (takes >10 hours) as

    java com/polytechnik/trading/QVM
    

    or three simple unit test of this paper algorithms:

    java com/polytechnik/trading/PnLInPsiHstateLegendreShifted\$PnLInPsiHstateLegendreShiftedTest
    
    java com/polytechnik/trading/PnLInPsiHstateLaguerre\$PnLInPsiHstateLaguerreTest
    
    java com/polytechnik/trading/PnLInPsiHstateMonomials\$PnLInPsiHstateMonomialsTest
    
  • com/polytechnik/freemoney/ The code implementing the theory of this paper. Most noticeable are:

    • com/polytechnik/freemoney/CommonlyUsedMoments.java Calculate the moments from (Time, Execution Price, Shares Traded) sequence of transactions using direct sampling.

    • com/polytechnik/freemoney/FreeMoneyForAll.java A wrapper to calculate the matrices Qj|f|Qk\Braket{Q_{j}}{f}{Q_{k}} from sampled moments Qmf\Braket{Q_{m}f}. Secondary sampling of Appendix D is also included.

    • com/polytechnik/freemoney/PFuture.java To calculate all the theory of this paper.

  • com/polytechnik/algorithms/ Drivers to call various algorithms.

Appendix D On Secondary Sampling

When direct sampling (66) of an observable is not available an advanced technique of ‘‘secondary sampling’’[2] can be applied to calculate the moments of it. An example. Assume on every tick a moving average is calculated. Then this calculated value is used as it were a new observable, and the moving average of this new observable is calculated. For trivial cases this gives nothing new: a moving average of a moving average is a moving average with different weight (for exponential moving average it is the moving average with twice lower exponent time). However, calculated quantity ‘‘as it were an observable’’ can be a characteristic that describes an immanent property of the system. In [2] we applied this technique to the maximal eigenvalue λ[IH]\lambda^{[IH]} of eigenproblem (10); we treated tick change in λ[IH]\lambda^{[IH]} (calculated value, see Fig. 1) as it were a change in the observable fl=λ[IH]|tlf_{l}=\lambda^{[IH]}\big{|}_{t_{l}} and calculated the moments with dfl=λ[IH]|tlλ[IH]|tl1df_{l}=\lambda^{[IH]}\left|{}_{t_{l}}-\lambda^{[IH]}\right|_{t_{l-1}} as Qmgdfdt=l=1MQm(x(tl))g(tl)ω(tl)dfl\Braket{Q_{m}g\frac{df}{dt}}=\sum_{l=1}^{M}Q_{m}(x(t_{l}))g(t_{l})\omega(t_{l})df_{l}. The simplest application of this ‘‘calculated observable’’ is the sum of price changes corresponding to positive λ[IH]\lambda^{[IH]} changes what gives the scalp price[2]:

𝒫\displaystyle{\cal P} =l=1M{p(tl)p(tl1) if λ[IH]|tlλ[IH]|tl1>00 otherwise\displaystyle=\sum_{l=1}^{M}\begin{cases}p(t_{l})-p(t_{l-1})&\text{\enspace if \quad$\lambda^{[IH]}\left|{}_{t_{l}}-\lambda^{[IH]}\right|_{t_{l-1}}>0$}\\ 0&\text{\enspace otherwise}\end{cases} (89)

Normalization 𝒫(tnow)=Plast{\cal P}(t_{now})=P^{last} is typically used for scalp price. Regular price corresponds to no condition on λ[IH]\lambda^{[IH]}. Whereas actual computation is performed in a single pass by highly optimized code using recurrent relation for moments and in-place calculation of the value to be used as a ‘‘new observable’’, for understanding the concept one may think about secondary sampling as having two-passes for input timeserie: first — scan all timeserie observations and build a ‘‘new observable’’ for every timeserie point read; second — scan this timeserie once again treating the value calculated on the first pass as it were a regular observable. This ‘‘secondary sampling’’ approach greatly extends the types of observable that can be studied. However, while being very powerful in calculation of moments that otherwise are not approachable at all, it has difficulties in interpretation of the results.

The implementation of this technique is available in \seqsplitcom/polytechnik/freemoney/CommonlyUsedMoments.java. The methods \seqsplitupdateWithSingleObservation recurrently adjusts the basis and adds calculated contributions corresponding to regular measures dPdP, dVdV, dAdA as in (66). After this call all regular moments become available, and we have an option to calculate the value of some ‘‘secondary’’ observable from them, such as λ[IH]\lambda^{[IH]}. When the calculation is completed — the method \seqsplitaddIHObservationSecondarySampling(double  IH) can be called. It, in addition to the moments already available from \seqsplitupdateWithSingleObservation, calculates, as in (66), three other moments corresponding to the measure dfl=IHlIHl1df_{l}=\texttt{IH}_{l}-\texttt{IH}_{l-1}, specifically: Qmdfdt\Braket{Q_{m}\frac{df}{dt}}, Qmpdfdt\Braket{Q_{m}p\frac{df}{dt}}, and Qmdpfdt\Braket{Q_{m}\frac{dpf}{dt}}. The IH can be a calculated characteristic of various meaning, and the moments of this characteristic are now obtained as it were a regular observable. This technique is also very convenient for unit tests of moments calculation by using regular observable as IH.

Appendix E On Separation of States Based On dI/dtdI/dt Sign

The value of future execution flow I0FI_{0}^{F} (19) allows us to obtain dI/dt\|dI/dt\| operator’s matrix elements. Using integration by parts (put p=constp=const in (72)) obtain:

ψ[j]missing|dIdt|ψ[k]\displaystyle\Braket{\psi^{[j]}\left missing}{\frac{dI}{dt}}{\psi^{[k]}} =I0Fω(x0)ψ[j](x0)ψ[k](x0)ED(ψ[j])|I|ψ[k]ψ[j]|I|ED(ψ[k])\displaystyle=I_{0}^{F}\omega(x_{0})\psi^{[j]}(x_{0})\psi^{[k]}(x_{0})-\Braket{\mathrm{ED}(\psi^{[j]})}{I}{\psi^{[k]}}-\Braket{\psi^{[j]}}{I}{\mathrm{ED}(\psi^{[k]})} (90)

This operator’s matrix elements cannot be obtained directly from sample (66), however the knowledge of the impact from the future allows us to apply an integration by parts. Actually this is the only operator for which integration by parts gives exact answer; for other operators (e.g d2p/dt2\|d^{2}p/dt^{2}\|) the boundary value at x0x_{0} is not known and matrix elements are typically obtained ‘‘within a boundary term’’. Only having determined the exact value of I0FI_{0}^{F} (that includes both: an ‘‘impact from the past’’ and an ‘‘impact from the future’’) it is possible to have the dI/dt\|dI/dt\| matrix elements that are accurate enough to consider an eigenproblem:

|dIdt|ψdI[i]\displaystyle\Ket{\frac{dI}{dt}\middle|\psi_{dI}^{[i]}} =λdI[i]|ψdI[i]\displaystyle=\lambda_{dI}^{[i]}\Ket{\psi_{dI}^{[i]}} (91)

Other than I0F=λ[IH]I_{0}^{F}=\lambda^{[IH]} (19) values can be used in the boundary term of (90), see [2] ‘‘Appendix E: On calculation of dI/dtdI/dt operator matrix elements from operator II’’ for a list of reasonable options for I0FI_{0}^{F}. The concept introduced in [2] is to treat low II \to high II and high II \to low II transitions separately, as they lead to a very different price behavior. These II-transitions correspond to dI/dtdI/dt derivative of different signs; corresponding operator dI/dt\|dI/dt\| always has eigenvalues λdI[i]\lambda_{dI}^{[i]} of different signs: ψ[IH]|dIdt|ψ[IH]=0\Braket{\psi^{[IH]}}{\frac{dI}{dt}}{\psi^{[IH]}}=0. Let us split the entire |ψ\Ket{\psi} space into direct sum of two subspaces888 Here we use dI/dt\|dI/dt\| matrix elements (90) that are calculated from directly sampled I\|I\| moments. An alternative is to split the states according to dI/dtdI/dt or dλ[IH]/dtd\lambda^{[IH]}/dt sign using the “secondary sampling” of [2]. The simplest example of it’s application is the scalp price 𝒫{\cal P} (89) that takes into account only “important” price changes (regular price is the sum of all price changes). . Construct two projection operators:

ΠdI+\displaystyle\|\Pi_{dI+}\| =i:λdI[i]>0|ψdI[i]ψdI[i]|\displaystyle=\sum\limits_{i:\,\lambda_{dI}^{[i]}>0}\Ket{\psi_{dI}^{[i]}}\Bra{\psi_{dI}^{[i]}} (92)
ΠdI\displaystyle\|\Pi_{dI-}\| =i:λdI[i]0|ψdI[i]ψdI[i]|\displaystyle=\sum\limits_{i:\,\lambda_{dI}^{[i]}\leq 0}\Ket{\psi_{dI}^{[i]}}\Bra{\psi_{dI}^{[i]}} (93)
𝟙\displaystyle\|\mathds{1}\| =ΠdI++ΠdI\displaystyle=\|\Pi_{dI+}\|+\|\Pi_{dI-}\| (94)

This transform can be considered as eigenvalues adjustment technique[12] where the eigenvalues (not the eigenvectors!) are adjusted for an effective identification of weak hydroacoustic signals. The ΠdI+\|\Pi_{dI+}\| can be viewed as dI/dt\|dI/dt\| operator with all negative eigenvalues set to 0 and all positive eigenvalues set to 11; the same with ΠdI\|\Pi_{dI-}\| for opposite sign. This technique is most easy to implement in (91) basis (where dI/dt\|dI/dt\| is diagonal), then to convert obtained projection operators back to the basis used applying (80). Alternatively one can convert all the matrices I\|I\|, pI\|pI\|, Vdpdt\|V\frac{dp}{dt}\|, dpIdt\|\frac{dpI}{dt}\|, ddtVdpdt\|\frac{d}{dt}V\frac{dp}{dt}\|, ρ\left\|\rho\right\| to the basis of (91) eigenproblem applying (81). All the results will be identical as the theory is gauge invariant[6]. With projection operators (92) and (93) any density matrix average can be written in the form:

Spurf|ρ\displaystyle\mathrm{Spur}\left\|f\middle|\rho\right\| =Spurf|ΠdI+|ρ+Spurf|ΠdI|ρ\displaystyle=\mathrm{Spur}\left\|f\middle|\Pi_{dI+}\middle|\rho\right\|+\mathrm{Spur}\left\|f\middle|\Pi_{dI-}\middle|\rho\right\| (95)

This split allows us to separate an average of ff in density matrix ρ\|\rho\| state to the ones corresponding to positive and negative dI/dtdI/dt.

First candidates on application of this technique are the terms from Total Lagrangian action (43) where the operators are calculated in the state of ρJIH\left\|\rho_{JIH}\right\| density matrix. Every Spur\mathrm{Spur} that enter into the Δ\Delta expression (44) can be split into dI/dtdI/dt contributions of different signs. There are several implementation of this technique, e.g. \seqsplitcom/polytechnik/freemoney/SplitdIdt.java and several others. The result, however, is not that great and this projection operators approach requires more research to be performed.

The property that requires special attention is that while the density matrix ρJIH\|\rho_{JIH}\| is obtained from the polynomial ψ[IH]2(x){\psi^{[IH]}}^{2}(x) with J(ψ[IH]2)J\left({\psi^{[IH]}}^{2}\right) transform (60), and all the average relations hold exactly, the density matrix itself may not have all the eigenvalues positive. This creates no problem with the total Spur\mathrm{Spur} but sometimes lead to spurious artifacts when combined with projection operators; the effect, however, is small. These small but negative eigenvalues of the density matrix, ‘‘Hermann Minkowski-style space’’, also require additional research.

E.1 Execution Flow Based Eigenvalues Adjustment Example

In the Appendix above we considered projection operators (94) to ‘‘split’’ I\|I\| or pI\|pI\| based on some other operator spectrum, e.g. dIdt\left\|\frac{dI}{dt}\right\|. To demonstrate a simplified example of this eigenvalues adjustment technique let us apply it to the operator I\|I\|. Consider the state ‘‘since |ψ[IH]\Ket{\psi^{[IH]}} till now’’ ρJIH\|\rho_{JIH}\| and a trading strategy: buy at execution flow below aggregated execution flow VIHTIH\frac{V_{IH}}{T_{IH}} with (110) and (111), and sell above it. The P&L position changes dSdS, see [3] Section ‘‘P&L operator and trading strategy’’, is:

dS\displaystyle dS =(IVIHTIH)dt\displaystyle=\left(I-\frac{V_{IH}}{T_{IH}}\right)dt (96)

Then the constraint 0=𝑑S0=\int dS is satisfied:

0\displaystyle 0 =SpurdSdt|ρJIH\displaystyle=\mathrm{Spur}\left\|\frac{dS}{dt}\middle|\rho_{JIH}\right\| (97)

and, for this dSdS, the P&L=p𝑑S\mathrm{P\&L}=-\int pdS can be calculated:

P&L\displaystyle\mathrm{P\&L} =SpurpdSdt|ρJIH\displaystyle=-\mathrm{Spur}\left\|\frac{pdS}{dt}\middle|\rho_{JIH}\right\| (98)

If we want to consider (98) as a superposition of |ψ[i]\Ket{\psi^{[i]}} states, introduce an operator:

Π\displaystyle\Pi =i=0n1|ψ[i](1VIHTIH1λ[i])ψ[i]|\displaystyle=\sum\limits_{i=0}^{n-1}\Ket{\psi^{[i]}}\left(1-\frac{V_{IH}}{T_{IH}}\frac{1}{\lambda^{[i]}}\right)\Bra{\psi^{[i]}} (99)
Π\displaystyle\Pi =Π++Π\displaystyle=\Pi_{+}+\Pi_{-} (100)
Π+\displaystyle\Pi_{+} =i:λ[i]>VIHTIHn1|ψ[i](1VIHTIH1λ[i])ψ[i]|\displaystyle=\sum\limits_{i:\,\lambda^{[i]}>\frac{V_{IH}}{T_{IH}}}^{n-1}\Ket{\psi^{[i]}}\left(1-\frac{V_{IH}}{T_{IH}}\frac{1}{\lambda^{[i]}}\right)\Bra{\psi^{[i]}} (101)
Π\displaystyle\Pi_{-} =i:λ[i]VIHTIHn1|ψ[i](1VIHTIH1λ[i])ψ[i]|\displaystyle=\sum\limits_{i:\,\lambda^{[i]}\leq\frac{V_{IH}}{T_{IH}}}^{n-1}\Ket{\psi^{[i]}}\left(1-\frac{V_{IH}}{T_{IH}}\frac{1}{\lambda^{[i]}}\right)\Bra{\psi^{[i]}} (102)

The operator Π\|\Pi\| is actually the operator I\|I\| but with the eigenvalues 1VIHTIH1λ[i]1-\frac{V_{IH}}{T_{IH}}\frac{1}{\lambda^{[i]}} instead of the λ[i]\lambda^{[i]}999 The operator Π\|\Pi\| may not correspond to a measure, i.e. it does not necessary correspond to m=02n2m=0\dots 2n-2 moments QmΠ\Braket{Q_{m}\Pi} from which to obtain Qj|Π|Qk\Braket{Q_{j}}{\Pi}{Q_{k}} using multiplication operator (53). . Then (97) and (98) become (103) and (104) respectively:

0\displaystyle 0 =SpurI|Π|ρJIH=SpurI|Π++Π|ρJIH\displaystyle=\mathrm{Spur}\left\|I\middle|\Pi\middle|\rho_{JIH}\right\|=\mathrm{Spur}\left\|I\middle|\Pi_{+}+\Pi_{-}\middle|\rho_{JIH}\right\| (103)
P&L\displaystyle\mathrm{P\&L} =SpurpI|Π|ρJIH=SpurpI|Π++Π|ρJIH\displaystyle=-\mathrm{Spur}\left\|pI\middle|\Pi\middle|\rho_{JIH}\right\|=-\mathrm{Spur}\left\|pI\middle|\Pi_{+}+\Pi_{-}\middle|\rho_{JIH}\right\| (104)

From these operators Π+\|\Pi_{+}\| and Π\|\Pi_{-}\| one can obtain ‘‘equilibrium prices’’ P±=SpurpI|Π±|ρJIHSpurI|Π±|ρJIHP^{\pm}=\frac{\mathrm{Spur}\left\|pI\middle|\Pi_{\pm}\middle|\rho_{JIH}\right\|}{\mathrm{Spur}\left\|I\middle|\Pi_{\pm}\middle|\rho_{JIH}\right\|} and etc. The result is similar to the technique of ‘‘extra volume’’ V~\widetilde{V} and PP^{*} of the Appendix F below; no advancing information we managed to obtain from (99).

Appendix F On The States Of Double Integration

The density matrix state ρJIH\rho_{JIH} (32) was obtained from the pure state of maximal execution flow |ψ[IH]\Ket{\psi^{[IH]}} by applying J()J(\cdot) transform (60) to the polynomial ψ[IH]2(x){\psi^{[IH]}}^{2}(x), a variant of integration by parts:

Spurdfdt|ρJIH=f|x0ρJIH(x0)ω(x0)ψ[IH]missing|f|ψ[IH]=f|x0ψ[IH]missing|f|ψ[IH]\displaystyle\mathrm{Spur}\left\|\frac{df}{dt}\middle|\rho_{JIH}\right\|=f\Big{|}_{x0}\rho_{JIH}(x_{0})\omega(x_{0})-\Braket{\psi^{[IH]}\left missing}{f}{\psi^{[IH]}}=f\Big{|}_{x0}-\Braket{\psi^{[IH]}\left missing}{f}{\psi^{[IH]}} (105)
j,k,l,m=0n1Qj(x0)Gjk1ρJIHklGlm1Qm(x0)=1\displaystyle\sum\limits_{j,k,l,m=0}^{n-1}Q_{j}(x_{0})G_{jk}^{-1}{\rho_{JIH}}_{kl}G_{lm}^{-1}Q_{m}(x_{0})=1 (106)

with (106) due to 1=ψ[IH]|ψ[IH]1=\Braket{\psi^{[IH]}}{\psi^{[IH]}} normalizing and 1=ω(x0)1=\omega(x_{0}) due to basis choice obtain familiar ‘‘integration by parts’’ relation (63). The ρJIH\left\|\rho_{JIH}\right\| density matrix allows us to calculate ‘‘execution-flow’’-related values from dIdt\left\|\frac{dI}{dt}\right\| operator. We already used this relation (a special case of (62)) to calculate e.g.

Spurdpdt|ρJIH\displaystyle\mathrm{Spur}\left\|\frac{dp}{dt}\middle|\rho_{JIH}\right\| =Plastψ|p|ψ\displaystyle=P^{last}-\Braket{\psi}{p}{\psi} (107)
SpurdpIdt|ρJIH\displaystyle\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JIH}\right\| =PlastI0Fψ[IH]missing|pI|ψ[IH]\displaystyle=P^{last}I_{0}^{F}-\Braket{\psi^{[IH]}\left missing}{pI}{\psi^{[IH]}} (108)
SpurdIdt|ρJIH\displaystyle\mathrm{Spur}\left\|\frac{dI}{dt}\middle|\rho_{JIH}\right\| =I0Fψ[IH]missing|I|ψ[IH]\displaystyle=I_{0}^{F}-\Braket{\psi^{[IH]}\left missing}{I}{\psi^{[IH]}} (109)
SpurI|ρJIH\displaystyle\mathrm{Spur}\left\|I\middle|\rho_{JIH}\right\| =VIH=Vlastψ[IH]missing|V|ψ[IH]\displaystyle=V_{IH}=V^{last}-\Braket{\psi^{[IH]}\left missing}{V}{\psi^{[IH]}} (110)
SpurρJIH\displaystyle\mathrm{Spur}\left\|\rho_{JIH}\right\| =TIH=Tlastψ[IH]missing|T|ψ[IH]\displaystyle=T_{IH}=T^{last}-\Braket{\psi^{[IH]}\left missing}{T}{\psi^{[IH]}} (111)

The (108) corresponds to (72), (109) with boundary condition (19) gives SpurdIdt|ρJIH=0\mathrm{Spur}\left\|\frac{dI}{dt}\middle|\rho_{JIH}\right\|=0 we used in (31), (110) and (111) are traded volume and time since |ψ[IH]\Ket{\psi^{[IH]}} spike till ‘‘now’’; typically we use normalizing Vlast=0V^{last}=0 and Tlast=0T^{last}=0.

Refer to caption
Figure 9: Price PP, price P[IH]P^{[IH]} (15), and PP^{*} (115) for AAPL stock on September, 20, 2012. The calculations in shifted Legendre basis with n=12n=12 and τ\tau=256sec.

Now consider a density matrix ρJJIH\rho_{JJIH} (33) obtained from the pure state of maximal execution flow |ψ[IH]\Ket{\psi^{[IH]}} by applying J()J(\cdot) transform (60) to the polynomial ψ[IH]2(x){\psi^{[IH]}}^{2}(x) twice. This density matrix corresponds to integration by parts performed twice. Obtain from (64):

SpurdIdt|ρJJIH\displaystyle\mathrm{Spur}\left\|\frac{dI}{dt}\middle|\rho_{JJIH}\right\| =I0FSpurρJIHSpurI|ρJIH=I0FTIHVIH\displaystyle=I_{0}^{F}\mathrm{Spur}\left\|\rho_{JIH}\right\|-\mathrm{Spur}\left\|I\middle|\rho_{JIH}\right\|=I_{0}^{F}T_{IH}-V_{IH} (112)
SpurdpIdt|ρJJIH\displaystyle\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JJIH}\right\| =I0FPlastSpurρJIHSpurpI|ρJIH\displaystyle=I_{0}^{F}P^{last}\mathrm{Spur}\left\|\rho_{JIH}\right\|-\mathrm{Spur}\left\|pI\middle|\rho_{JIH}\right\| (113)

The ρJJIH\left\|\rho_{JJIH}\right\| density matrix allows us to calculate ‘‘volume’’-related values from dIdt\left\|\frac{dI}{dt}\right\| operator. One of the major results of this paper is established in Section IV.2 fact that in ρJIH\left\|\rho_{JIH}\right\| state the values of operators pdIdt\left\|p\frac{dI}{dt}\right\| and Idpdt\left\|I\frac{dp}{dt}\right\| are very close and their difference (if exists) gives future price (36). Let us consider these operators not in ρJIH\left\|\rho_{JIH}\right\| state, but instead in the state ρJJIH\left\|\rho_{JJIH}\right\|. An important difference from ρJIH\left\|\rho_{JIH}\right\| state is that the condition of dIdt\left\|\frac{dI}{dt}\right\| being zero in ρJJIH\left\|\rho_{JJIH}\right\| state no longer holds, as execution flow I=dV/dtI=dV/dt and aggregated execution flow VIH/TIHV_{IH}/T_{IH} are different in |ψ[IH]\Ket{\psi^{[IH]}} state. The (112) requires an ‘‘extra volume’’ V~\widetilde{V}

V~\displaystyle\widetilde{V} =(I0FVIHTIH)TIH\displaystyle=\left(I_{0}^{F}-\frac{V_{IH}}{T_{IH}}\right)T_{IH} (114)

to obtain proper value of operator pdIdt\left\|p\frac{dI}{dt}\right\|; an alternative is to use eigenvalues adjustment technique of Appendix E.1 above. Using (35) obtain

Δ\displaystyle\Delta =SpurIdpdt|ρJJIHSpurpdIdt|ρJJIH+V~P\displaystyle=\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JJIH}\right\|-\mathrm{Spur}\left\|p\frac{dI}{dt}\middle|\rho_{JJIH}\right\|+\widetilde{V}P^{*}
=2SpurIdpdt|ρJJIHSpurdpIdt|ρJJIH+V~P\displaystyle=2\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JJIH}\right\|-\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JJIH}\right\|+\widetilde{V}P^{*} (115)

The difference from (35) is that there is an extra term V~P\widetilde{V}P^{*} caused by the difference between I0FI_{0}^{F} and VIHTIH\frac{V_{IH}}{T_{IH}}. We can consider |V|ψ=λ|T|ψ\Ket{V|\psi}=\lambda\Ket{T|\psi} providing equal execution flow and aggregated execution flow, see [2]: ‘‘Appendix C: The state of maximal aggregated execution flow V/TV/T’’, but these states gives little improvement. Consider instead a simplistic approach: select the value of PP^{*} that makes Δ\Delta equals to zero:

P\displaystyle P^{*} =1V~(2SpurIdpdt|ρJJIHSpurdpIdt|ρJJIH)\displaystyle=-\,\frac{1}{\widetilde{V}}\left\lgroup 2\mathrm{Spur}\left\|I\frac{dp}{dt}\middle|\rho_{JJIH}\right\|-\mathrm{Spur}\left\|\frac{dpI}{dt}\middle|\rho_{JJIH}\right\|\right\rgroup (116)

In Fig. 9 the PP^{*} is presented. We see no ‘‘advancing’’ property as it is for (36), this is an indicator of ‘‘lagging’’ type. This makes us to conclude that the state ρJJIH\rho_{JJIH} while it has a number of interesting properties to research, does not immediately provide and ‘‘advancing’’ indicator. The ρJIH\rho_{JIH} is probably the only state in which pdIdt\left\|p\frac{dI}{dt}\right\| and Idpdt\left\|I\frac{dp}{dt}\right\| operators are very close and their difference (if exists) gives future price (36).

References