How Robust are Limit Order Book Representations
under Data Perturbation?

The limit order book (LOB) is used by financial exchanges to match buyers and sellers of a particular instrument and acts as an indicator of the supply and demand at a given point in time. It can be described as a self-evolving process with complex spatial and temporal structures revealing the price dynamics at the microstructural level. Market making, optimal execution and statistical arbitrage strategies, all require a good understanding of the LOB and its dynamics. Figure 1 (A) shows a snapshot of the LOB with both the bid (buy) and ask (sell) order volumes accumulated at each price level. The mid-price is the average of the best (lowest) ask price and the best (highest) bid price and the difference between them is referred to as the bid-ask spread. The LOB gets updated continuously with order placements, cancellations and executions.

The use of algorithmic trading strategies and the digitisation of exchange activities has made available a tremendous amount of LOB data for practitioners and researchers to study the market dynamics from data-driven approaches. This led to a surge in interest for big data applications in the financial markets and machine learning (including deep learning) models becoming a trend in the quantitative finance domain (Buehler et al., 2019), (Wiese et al., 2020). The LOB data come in different degrees of granularity with L1 data providing the best bid/ask prices and volumes, L2 data providing the same data across all price levels and L3 data containing the non-aggregated orders placed by market participants.

In our work, we focus on how LOB data is typically represented by taking a price forecasting task as an example. In practice, a vector representation of the raw limit order book information is needed for upcoming learning processes. This transformation from raw data to feature vectors is typically referred to as feature engineering, which requires a good and comprehensive understanding of the domain knowledge to make sure the extracted features match the learning task. By contrast, representation learning, also called feature learning, is an automated approach to discover an optimal representation for the data. The major difference between feature engineering and representation learning is whether the representation is formed in a purely data-driven way. Also, it is common for a machine learning system to involve both feature engineering and representation learning with multiple levels of representation appearing at different stage of processing (see figure 1 (B)).

The performance of machine learning models is heavily influenced by the data representation scheme (Bengio et al., 2013). For neural networks, the representation learning and the prediction processes are combined within the network structure and are trained together towards the same target function. In this case, the original representation of LOB, i.e. the input representation to neural networks, becomes the foundation of the entire model. Presently, the price level-based data representation scheme is used in almost all recent studies (Tsantekidis et al., 2017b, a; Tran et al., 2018; Zhang et al., 2019; Mahfouz et al., 2019; Sirignano, 2019; Tsantekidis et al., 2020; Wallbridge, 2020) applying deep learning models on LOB data. However, this representation scheme is rarely discussed or investigated towards its compatibility with machine learning especially deep learning models. In this paper, we propose a pioneer insight to challenge this level-based LOB representation for machine learning models, by showing potential risks under subtle perturbations and raising concerns regarding to its robustness.

We first introduce the commonly-used level-based LOB representation scheme found in benchmark and research datasets (e.g. (Ntakaris et al., 2018; Huang & Polak, 2011)). The spatial representation of this scheme, i.e. an LOB snapshot, is a vector $s_{t}=\left\{p_{a}^{i}(t),v_{a}^{i}(t),p_{b}^{i}(t),v_{b}^{i}(t)\right\}_{i=1}^{L}$, where, $p_{a}^{i}(t)$, $p_{b}^{i}(t)$ are the ask and bid prices for price level $i$ and $v_{a}^{i}(t)$, $v_{b}^{i}(t)$ are the ask and bid volumes respectively. Temporally, the history of the LOB snapshots is stacked to reflect an evolution of the market, leading to the commonly-used limit order book data structure $S\in\mathbb{R}^{T\times 4L}$ where T is the history length and L is the amount of price levels considered for each side.

This level-based representation is efficient and convenient from the perspective of human understanding and how the matching engine in exchanges works. From the machine learning perspective, this representation has some particular characteristics. The most intuitive one is that the price and volume for each LOB level are tied together - any disentanglement or distortion to this would result in an invalid representation. In addition, the spatial structure across different levels is not homogeneous since there is no assumption for adjacent price levels to have fixed intervals. Note that, homogeneous spatial relationship is a basic assumption for convolutional neural networks (CNN) due to the parameter sharing mechanism. Thus, the heterogeneous spatial feature of level-based LOB data may reduce model robustness when learning with CNN models. From the temporal perspective, we also realise some instability of the representation due to occasional shifts of price levels - the previous best bid/ask data can suddenly shift to second best bid/ask channel if a new order is placed with a better price.

We present a simple data perturbation method to examine the robustness of the price level-based representation from the machine learning perspective. In some LOB data for equities, the price difference between adjacent price levels is sometimes larger than the tick size (the minimum price increment change allowed). This is especially prevalent in small-tick stocks and can result in the entire LOB shifting even if a small order of the minimum allowable size is placed at a price in between the existing price levels. The data perturbation method presented assumes that the data is perturbed by small size orders at empty price levels beyond the best ask/bid prices. This perturbation ensures no change is made to the mid-price before and after perturbation to make sure the prediction labels are not affected.

We illustrate this data perturbation with a synthetic LOB example as shown in Fig. 2. Fig. 2 (A) shows the synthetic LOB snapshot with 10 price levels in both ask and bid sides of the LOB (marked as L1-L10) before any perturbation. We assume the tick size is 0.01 and the minimum order size present in our data is 1. In this LOB snapshot, the mid-price is 10.00 with bid-ask spread equal to 0.04. We can observe some price levels where no orders are placed, such as 10.03, 10.06 in the ask side and 9.96, 9.94 in the bid side. To perturb this LOB data, one can place orders with allowed minimum order size to fill these empty price levels. These minimum size orders may seem to be not influential since 1) they do not effect the mid price, 2) their volumes are tiny. c (see Fig. 2 (B)). Approximately half of the original price level information is no longer visible after perturbation (e.g. ask-side L5 to L10 information is not included in representation after perturbation) and while the rest are preserved, they are shifted to different levels in the LOB representation (e.g., the ask-side L2 appears in ask-side L3 after perturbation).

Intuitively, this perturbation has two impacts from the machine learning point of view. Firstly, it shifts the 40-dimensional input space dramatically. For example, the Euclidean distance between these two 40-dimensional vectors before and after perturbation is 344.623 whereas actually the total volume of orders applied is only 10. This means that the level-based representation scheme does not bring local smoothness. Furthermore, it narrows the scope of vision of machine learning models to ‘observe’ the market. As shown in the LOB data visualisation plot in Fig. 2, the gray areas are masked out for the model input after perturbation.

The study of the importance of robust data representation, the criteria for evaluating the quality of the representation, and the variety of methods for learning these representations is studied extensively in the machine learning literature with (Bengio et al., 2013) providing a survey of these methods. In our work, we focus on the representation of financial market microstructure data. (Bouchaud et al., 2018), (Abergel et al., 2016) study the structure and empirical properties of limit order books and provide a set of statistical properties (referred to as stylized facts) using NASDAQ exchange data. On the other hand, (Lehalle & Laruelle, 2018) discusses the practical aspects and issues of market structure, design, price formation, discovery and the behaviour of different actors in limit order book markets. A significant amount of research in recent years focused on applying deep learning models on limit order book data for the purposes of price forecasting or price movements classification. Different model architectures were investigated including multi-layer perceptrons (Mahfouz et al., 2019), recurrent neural networks (Sirignano & Cont, 2019), convolutional neural networks (Zhang et al., 2019), (Zhang et al., 2018) and self-attention transformer networks (Wallbridge, 2020).

In this section, we implement a series of experiments to examine the robustness of the LOB representation under different data perturbations. We take price forecasting as the task paradigm and train various machine learning models to perform such task and examine their performance when encountering unexpected perturbation.

In this paper, we discussed the importance of data representations to machine learning models applied to LOB-related tasks. We designed data perturbation scenarios to test the robustness of commonly-used machine learning models with the level-based LOB representation scheme in price forecasting tasks. We show that, although the perturbation is subtle, it still shows a large impact to the level-based representation and thus leads to significant performance decay. In particular, this performance decay is more severe in sophisticated machine learning models.

Based on the findings in this paper and our understanding of representations, we would like to raise some desiderata for improving the robustness of LOB-related data representations and machine learning models designed on top of certain representations:

• •

  Smoothness: LOB Data representations should not change dramatically under subtle perturbations.

• •

  Efficiency: LOB data representation should organise data in a efficient manner to reduce the curse of dimensionality.

• •

  Stochasticity: LOB data representation should consider the stochasticity of the market, instead of treating the market process as a deterministic one.

• •

  Validity: Basic assumptions needs to be matched between data representations and learning models. If not, these models may contain unknown risks due to invalid fundamental settings.

Our future work would focus on feature engineering and representation learning for LOB data that can fulfill these desiderata and combining LOB representations with various machine learning tasks including forecasting, reinforcement learning and etc.

The authors would like to acknowledge our colleagues Vacslav Gluckov, Rui Silva, Thomas Spooner and Jeremy Turiel and for their input and suggestions at various key stages of the research.

This paper was prepared for informational purposes by the Artificial Intelligence Research group of JPMorgan Chase & Co and its affiliates (”J.P. Morgan”), and is not a product of the Research Department of J.P. Morgan. J.P. Morgan makes no representation and warranty whatsoever and disclaims all liability, for the completeness, accuracy or reliability of the information contained herein. This document is not intended as investment research or investment advice, or a recommendation, offer or solicitation for the purchase or sale of any security, financial instrument, financial product or service, or to be used in any way for evaluating the merits of participating in any transaction, and shall not constitute a solicitation under any jurisdiction or to any person, if such solicitation under such jurisdiction or to such person would be unlawful.