A combinatorial view of stochastic processes: White noise

The incorporation of stochastic ingredients in models describing phenomena in all disciplines is now a standard in scientific practice. White noise is one of the most important of such stochastic ingredients. Although tools for identifying white and other types of noise exist [1, 2], there is a permanent demand for reliable and robust statistical methods for analyzing data in order to distinguish noise and filter it from signals in experiments. Or in hypothesis tests, for assessing the plausibility of the outcome of an experiment being the result of randomness and not a significant, controllable effect. Due to its ubiquity in experiments and its mathematical simplicity, white noise is very often the most convenient stochastic component that adds realism to a dynamic model, commonly regarded as the noise polluting observations. It can be continuous or discrete both in time or in distribution, so it can be applied to many scenarios. It is a stationary and independent and identically distributed process, all relatively simple properties for a stochastic process. Here we present a combinatorial perspective to study white noise inspired in the concept of ordinal patterns. An ordinal pattern of length $n$ is the diagramatic representation of the inequality fulfilled by a subsequence of $n$ points $x_{1},…,x_{n}$ in a time series $\{x_{t}\}_{t\in I}$. We discuss ordinal patterns in detail in Sec. II. This concept was used in 2002 by Bandt and Pompe [3] for building a measure of complexity for time series named Permutation Entropy (PE). PE has proven its value not only in applications, when used to analyze time series from a great variety of phenomena [4, 5], but it is also of theoretical relevance since it is equivalent to the Kolmogorov-Sinai entropy for a large class of piece-wise continuous maps of the interval [6, 7]. The procedure for computing PE consists in first choosing the size $n$ for the window that will be used in the analysis, corresponding to the embedding dimension in a Takens embedding [2]. Then we slide the window through the series (equivalent to construct the lagged or delay vectors [2]), to find the frequencies $\pi_{j},k=1,…,n!$ with which ordinal patterns occur. Then we compute the associated Shannon entropy $H_{\pi}=-\sum_{j}\pi_{j}\log\pi_{j}$. For white noise in the long time series limit all the patterns are equally likely, therefore their frequencies follow a discrete uniform probability mass function (p.m.f.) with support on the integers $1,…,n!$, which is equivalent to the probability measure of permutations over the symmetric group of degree $n$, $S_{n}$. Despite its relevance and wide range of applications, there are few rigorous studies on the properties of PE for its use in statistical inference. To the best of the author’s knowledge, this is addressed only in works such as [8], where the authors investigate the expectation value and variance of PE for finite time series of white noise, and later the same authors address the effect of ordinal pattern selection on the variance of PE [9]. In the customary PE approach every permutation is, in a sense, considered as a class, since the count of every single permutation is important. The effect of this is that the empirical distributions obtained from finite-length observations will be very sensitive to relatively minor changes in the proportions of each observed pattern [8]. This lack of robustness represents a liability when trying to distinguish noise from structure. Another problem is the factorial growth of the number of classes, enlarging the discrete support correspondingly and making the analysis both impractical and meaningless for values of the embedding dimension beyond low or moderate $n$ ($\sim 10$), since the required length of a time series that will have a chance to display roughly one representative member of each class would be on the same order of $n!$ (already around 3 million observations for $n=10$).

In this work, we address these limitations by introducing a new statistic for permutations in Sec. III. This statistic is a functional over the symmetric group $S_{n}$, that can be interpreted as a measure of asymmetry for ordinal patterns. The functional divides the symmetric group into classes corresponding to a coarse notion of levels of overall increasing or decreasing behaviour of a pattern. In turn, this results on the transformation of the original discrete uniform probability measure of the patterns over $S_{n}$, into a new probability measure that concentrates around its expected value, as we show in Sec IV. This has practical and conceptual consequences, such as the ability of performing a suitably modified version of the ordinal pattern analysis for very large embedding dimensions, since now the number of classes of patterns to be tracked is reduced from $\mathcal{O}(n!)$ to merely $\mathcal{O}(n^{2})$. The probability mass functions corresponding to our functional can be approximated by a Gaussian distribution, that is itself an exponential family. This guarantees the existence of natural sufficient statistics for the estimation of our statistic, as we explain in Sec IV. We open Sec. V with an illustration of our framework by analysing white noise from different source distributions and discussing its potential for distinguishing the deterministic signature of a chaotic orbit from a discrete map by inspection of the empirical p.m.f. obtained from our modified pattern analysis. Then we take experimental 3D tracks of gold nanoparticles and design a test for identifying the statistical independence among the spatial increments on each coordinate in a plane of observation. We finish by discussing our results and making additional remarks on the advantages and drawbacks of the presented framework.

A permutation $\tau$ is a bijection $\tau:S\rightarrow S$. If we take the set $S$ to be the sequence of integers $S=\{1,2,3,\ldots,n\}$, then $\tau(i)=\tau_{i}$ maps this sequence into itself $\tau_{i}=k,\ \ i,k=1,2,3,\ldots,n$. Due to its bijective nature, we call the arrangement $\tau=\tau_{1}\tau_{2}\cdots\tau_{n}$ also a permutation of length $n$. Alternatively we can call it a word produced by $\tau$ on $n$ symbols. The set $S_{n}=\{\tau:\tau(i)=k,\ \ i,k=1,2,3,\ldots,n\}$ together with the operation of functional composition is the symmetric group on $n$ symbols, of cardinality $|S_{n}|=n!$ [10]. We will denote the set of symbols as $[n]:=\{1,2,\ldots,n\}$ and an interval of symbols is denoted by $[i,j]\in[n]$.

Consider the lattice $L_{n}$ formed by the cartesian product $S_{n}\times S_{n}$. An ordinal pattern [3], that we will denote here as $d_{\tau}=\langle\tau(1),\tau(2),\ldots,\tau(n)\rangle$ or simply $d$ is a directed graph in $L_{n}$ that connects ordered pairs that are consecutive in their first index $(i,\tau(i))\rightarrow(i+1,\tau(i+1))$, $i=1,2,\ldots,n$, hence $d:S\times S\rightarrow S_{n}$. The angle brackets $\langle\cdot\rangle$ explicitly indicate that the elements of the $n$-tuple $(\tau(1),\tau(2),\ldots,\tau(n))$ are connected consecutively conforming a graph.

For instance, the graph $(1,1)\rightarrow(2,2)$, or $\langle 1,2\rangle$ in the $L_{2}$ lattice is an ordinal pattern, since $\tau(1)=1,\tau(2)=2$, then $d_{\mathrm{id}}=\langle 1,2\rangle$ corresponds to the identity permutation in $S_{2}$, $\tau_{\mathrm{id}}=12$. The graph $(1,1)\rightarrow(2,1)=\langle 1,1\rangle$ is not a valid diagram since the word $11$ is not a permutation, i.e. $11\notin S_{2}=\{12,21\}$. The set of all diagrams is denoted by $D_{n}=\{d:d=\langle\tau(1),\tau(2),\ldots,\tau(n)\rangle\}$. Tables 2 and 3 display the permutations and their corresponding graph sets $D_{3}$, $D_{4}$, respectively.

All of the permutation statistics displayed in Table I reflect different symmetries in $S_{n}$. The sign of the permutation is arguably the most basic statistic, telling explicitly whether the number of flips of symbols in a permutation $\tau$ relative to the identity $\tau_{\mathrm{id}}=123\cdots n$ is even ($\operatorname{sgn}(\tau)=+1$) or odd ($\operatorname{sgn}(\tau)=-1$). The inversion number explicitly counts the pairs for which the symbol in position $i$ is larger than the one at $i+1$. Summing up the indices of the larger symbols in each inversion yields the major index $\mathrm{maj}(\tau)=\sum_{\tau(i)>\tau(i+1)}i$.

The construction of the set of ordinal patterns from a realization of any process $X_{t}$, consists on mapping portions of that process time series $\{x_{t}\}_{t\in I}$ (with $I=[L]$ an index set, usually $[N]={1,2,\ldots,L}$), to the diagrams in Sec.II by sliding a window of size $n$ over $\{x_{t}\}_{t\in I}$. We can inspect larger portions of the series without increasing $n$ by skipping (lagging) a number of $l-1$ points after every choice on each window, so the number $l$ is correspondingly known as lag. For instance, a lag of $l=1$ means that we do not skip any point and we slide the window without gaps. The collection of windows of $n$ points are called delay or lagged vectors, and the process of construction of these vectors is known as an embedding [2]. Correspondingly, the natural number $n$ indicating the window size is called embedding dimension. This terminology comes from the embedding theorems that form the basis of the state space reconstruction methods from scalar time series, known in general as time delay embeddings [2]. Hence, for the ordinal pattern analysis, we first make the embedding of the time series for a chosen dimension $n$ and lag $l$. This yields a total of $N=L-(n-1)l$ lagged vectors. Then we rank the amplitudes $\{x_{1},x_{2},\ldots,x_{n}\}$ on every lagged vector according to their magnitude, thus obtaining the ranked vectors $\{x_{\tau(1)},x_{\tau(2)},\ldots,x_{\tau(n)}\}$. The sequence of indices in these ranked vectors are the ranking permutations $\tau=\tau_{1}\tau_{2}\cdots\tau_{n}$, where $\tau_{i}=\tau(i)$. Finally, these permutations have associated ordinal patterns as we saw in Sec. II. In this way the local information on the relative ordering is preserved, and as a consequence also the relevant information on the correlation structure of the series. However, a simple but careful inspection of this mapping makes evident that the relative ordering is not the only information preserved. In fact, if we interpret the ranks as abstract weights assigned to the amplitudes of the process, then it is clear that also information on the overall variation within portions of these windows of $n$ data points is preserved. Questions such as the weight accumulated in sub-intervals within each bin of size $n$ arise naturally from this observation. An equally natural choice of sub-interval for analysis within the $n$-window is half the bin, so as to build a measure to estimate the tendency of the process to have local increasing, decreasing or close to constant behavior. This is in close analogy to the concept of derivative, but getting rid of the information on the specific amplitudes that the process takes. In order to study this asymmetry on the total weight concentrated on each half of a length $n$ window, we provide the following

Definition 1 Functional $\alpha(\tau)$. For every permutation $\tau\in S_{n}$, $n$ a non-negative integer, define the functional

$$\alpha(\tau)=\begin{cases}\sum_{i=m+2}^{n}\tau(i)-\sum_{i=1}^{m}\tau(i),\ n\ \mathrm{odd}\\ \sum_{i=m+1}^{n}\tau(i)-\sum_{i=1}^{m}\tau(i),\ n\ \mathrm{even},\end{cases}$$

(5)

where $m\equiv\lfloor{\frac{n}{2}\rfloor}$.

The functional thus defined is invariant under a shift of the sequence $[n]$ by an integer. This transformation is also invariant under monotonic transformations of the process $X_{t}$, since the relative ordering is not affected. The case for odd $n$ gives mixed statistical behavior, since the middle permutation symbol $\tau_{m+1}$ is ignored in the computation of $\alpha(\tau)$, implying that for a given permutation lenght $n$, when the ignored symbol happens to be $\tau_{m+1}=n$, there will be $(n-1)!$ permutations that display the same statistics as in the full problem for permutation lenght $n-1$. By the invariance of $\alpha(\tau)$ under a shift of $\{1,2,\ldots,n\}$ by an integer, when the middle symbol is $\tau_{m+1}=1$ we have the same situation as before. Other ignored symbols produce different and more complicated effects. Therefore, we shall limit here to the case for even $n$, which reduces Eq. (5) to

$$\alpha(\tau)=\sum_{k=1}^{m}\left[\tau\left(k+m\right)-\tau\left(m-k+1\right)\right],\ \ m=\frac{n}{2}.$$

(6)

The definition of $\alpha(\tau)$ allows for an obvious source of degeneracy or multiplicity, since the order of the summands on each half $s_{l},s_{r}$ is irrelevant. Let us denote the number of permutations that share the same value of $\alpha(\tau)$ by $\phi(\alpha)$. For every distinct value of $\alpha(\tau)$, there are (by shuffling the terms on each sum $s_{l},s_{r}$) at least $|L|\cdot|R|=\left(m!\right)^{2}\equiv\eta$ reorderings that share the same value, hence $\phi(\alpha)\geq\eta$. For instance, if we consider $\tau_{\mathrm{id}}$ we get $\alpha(\tau_{\mathrm{id}})=\alpha_{M}$ as above, which has the least degeneracy $\phi(\alpha_{M})=\eta$. The next possible value of $\alpha(\tau)$ can be obtained from $\tau_{\mathrm{id}}=123\cdots n$ by switching $m$, the largest symbol in $L$, with the smallest symbol in $R$, which is $m+1$. This clearly increases the value of the left partial sum by one $s_{l}\rightarrow s_{l}+1$ while decreasing the right partial sum by one unit $s_{r}\rightarrow s_{r}-1$, with the effect of decreasing $\alpha(\tau_{\mathrm{id}})$ by two units $\alpha(\tau_{\mathrm{id}})\rightarrow\alpha(\tau_{\mathrm{id}})-2$, then the next value is $\alpha^{\prime}=\alpha_{M}-2$. Proceeding in this way, we get the set of all possible $\alpha$-values over $S_{n}$ (for even $n$)

$$A_{n}=\{-\alpha_{M},-\alpha_{M}+2,-\alpha_{M}+4,\ldots,\alpha_{M}-4,\alpha_{M}-2,\alpha_{M}\}$$

(7)

	$\displaystyle a_{i}(2)$	$\displaystyle=$	$\displaystyle 1,1$
	$\displaystyle a_{i}(4)$	$\displaystyle=$	$\displaystyle 1,1,2$
	$\displaystyle a_{i}(6)$	$\displaystyle=$	$\displaystyle 1,1,2,3,3$
	$\displaystyle a_{i}(8)$	$\displaystyle=$	$\displaystyle 1,1,2,3,5,5,7,7,8$
	$\displaystyle a_{i}(10)$	$\displaystyle=$	$\displaystyle 1,1,2,3,5,7,9,11,14,16,18,19,20$
	$\displaystyle a_{i}(12)$	$\displaystyle=$	$\displaystyle 1,1,2,3,5,7,11,13,18,22,28,32,39,42,48,$
			$\displaystyle 51,55,55,58$
	$\displaystyle a_{i}(14)$	$\displaystyle=$	$\displaystyle 1,1,2,3,5,7,11,15,20,26,34,42,53,63,75,$		(8)
			$\displaystyle 87,100,112,125,136,146,155,162,166,169$
		$\displaystyle\vdots$

It is clear that $\sum_{i}\phi(\alpha_{i})=\eta\sum_{i}a_{i}=n!$, therefore,

$$\sum_{i=0}^{m^{2}}a_{i}={{n}\choose{m}},\ \ m=\frac{n}{2}.$$

(9)

Theorem (Bóna 2012 [10]) Let $n$ and $k$ be fixed non-negative integers so that $k\leq n$. Let $b_{i}$ denote the number of $k$-element subsets of $[n]$ whose elements have sum ${k+1\choose 2}+i$, that is, $i$ larger than the minimum. Then we have

$${\mathbf{n}\brack\mathbf{k}}:=\sum_{i=0}^{k(n-k)}b_{i}q^{i}.$$

(10)

In other words, ${\mathbf{n}\brack\mathbf{k}}$ is the ordinary generating function of the $k$-element subsets of $[n]$ according to the sum of their elements.

$${\mathbf{n}\brack\mathbf{k}}=\frac{[\mathbf{n}]!}{[\mathbf{n-k}]![\mathbf{k}]!}=\frac{(1-q^{n})(1-q^{n-1})\cdots(1-q^{n-k+1})}{(1-q)(1-q^{2})\cdots(1-q^{k})}$$

(11)

$$[\mathbf{k}]=\frac{1-q^{k}}{1-q}=1+q+q^{2}+\cdots+q^{k-1},\ \ q\neq 1,$$

(12)

	$\displaystyle[\mathbf{k}]!$	$\displaystyle=$	$\displaystyle[\mathbf{1}][\mathbf{2}][\mathbf{3}]\cdots[\mathbf{k}]$
		$\displaystyle=$	$\displaystyle(1+q)(1+q+q^{2})\cdots(1+q+q^{2}\cdots+q^{k-1}).$
					(13)

$$G_{n}(q)=\sum_{i=0}^{m^{2}}a_{i}q^{i},\ \ a_{i}=a_{m^{2}-i}>0,\ \ q\neq 1,$$

(14)

$$a_{i}=[q^{i}]{\mathbf{n}\brack\mathbf{m}}=[q^{i}]G_{n}(q)$$

(15)

$$P_{\chi_{n}}(\alpha_{i})=\frac{a_{i}}{\sum_{j=0}^{m^{2}}a_{j}}=\frac{[q^{i}]{\mathbf{n}\brack\mathbf{m}}}{{n\choose m}},$$

(16)

	$\displaystyle G_{\chi_{n}}(q)$	$\displaystyle=$	$\displaystyle\sum_{0\leq i\leq m^{2}}p_{i}q^{i}$		(17)
		$\displaystyle=$	$\displaystyle\frac{G_{n}(q)}{G_{n}(1)}$		(18)
		$\displaystyle=$	$\displaystyle\frac{{\mathbf{n}\brack\mathbf{m}}}{{n\choose m}}$		(19)

By applying the functional $\alpha(\tau)$ defined in Eq. (6) to the set of observed permutations obtained from embedding a stochastic process $X_{t}$ as explained previously, we get an associated stochastic process in terms of $\alpha$. The functional $\alpha(\tau)$ can be applied to any process, but here we limit to its use for the analysis of white noise processes.

A white noise process $X_{t}$ is a continuous or discrete time stochastic process with the following properties

	$\displaystyle E[X_{t}]=0,\ \mathrm{and}\ \mathrm{Var}[X_{t}]=\sigma^{2},\ \mathrm{for\ all\ }t\in\Omega$		(20)
	$\displaystyle E[X_{t}X_{s}]=\sigma^{2}\delta(t-s),\ \mathrm{for\ all}\ s,t\in\Omega$		(21)

where $0<\sigma^{2}<\infty$, $\Omega$ is the set in which $s,t$ take values, for instance $\Omega=\mathbb{R}$ for a continuous process. From these properties, the zero-mean and finite variance conditions are irrelevant for the application of $\alpha(\tau)$, since they merely are information about the scale of the process, to which our functional is blind.

The values of $\alpha$ over an embedding in a process are well defined, so long as the source process $X_{t}$ has a continuous support (later we will see that this condition can be sometimes relaxed), thus having a zero probability of observing repeated amplitudes of the process $X_{t}$. This means that the ranking permutations are well defined. Of course, real observations have a finite resolution, but even considering this, repetition of amplitudes from a stochastic process with continuous support would be expected to be highly unlikely at least in a finite time. Considering the former facts, we arrive at the following

Definition 2 Induced $\chi$-process and $\alpha$-series. For every choice of non-negative integers $n$, $m$ and $l$ (with $n=2m$) and every white noise process $X_{t}$, the functional in Eq. (6) induces the discrete-time process $\chi_{k}$ with discrete support $A_{n}$ (7). Correspondingly, we denote a realization or time series of the process $\chi_{k}$ as

$$\{\alpha_{k}\}_{k\in[N]},$$

(22)

where $[N]=\{1,2,\ldots,N\}$, $L$ is the length of an observed realization of $X_{t}$, and $N=L-(n-1)l$, is the number of delay vectors in an embedding with fixed values $n$ and $l$.

In general, for arbitrary values of $n$ and $l$, the process $\chi_{k}$ and, consequently, $\{\alpha_{k}\}_{k\in[N]}$ are correlated due to the overlap of the embedding vectors, that is in turn reflected in a sequence of permutations that are not independent. Nevertheless, the process could become effectively uncorrelated for some values of $l$ or at large values of both $n$ and $l$. In order to visualize how the p.m.f $P_{\chi_{n}}(\alpha_{i})$ changes as $n$ increases, let us plot Eq. (16) for different $n$ values. To facilitate comparisons, we make use of the standardized variable $Z_{\alpha}=(\chi-\mu_{\chi})/\sigma_{\chi}$. Since $\mu_{\chi}=0$, we have the simple quotient

$$Z_{\alpha}=\frac{\chi}{\sigma_{\chi}}.$$

(23)

Let us come back briefly to the reflections of diagrams introduced in Sec. II to add understanding to $P_{\chi}(\alpha)$. As illustrated in Figs. 1 and 1, reflecting the diagrams change the corresponding values of $\alpha$ up to sign only. This means that for every $d\in D_{n}$ the compositions of $h$ and $v$ fulfill

$$\alpha[(h\circ v)(d)]=\alpha[(v\circ h)(d)],$$

(24)

	$\displaystyle\alpha(d_{\tau})$	$\displaystyle=$	$\displaystyle-\alpha[v(d_{1})]$
		$\displaystyle=$	$\displaystyle\alpha[hv(d_{1})]$
		$\displaystyle=$	$\displaystyle\alpha[d_{2}]$
		$\displaystyle=$	$\displaystyle 5$

By direct computation of the variance $\sigma_{\chi_{n}}^{2}=\sum_{0\leq i\leq m^{2}}\alpha_{i}^{2}p_{i}$ for succesive $n$ values we arrive at the formula

$$\sigma_{\chi}^{2}=\frac{m^{2}(2m+1)}{3}$$

(25)

which for $m\rightarrow\infty$ becomes

$$\sigma_{\chi}^{2}=\frac{2}{3}m^{3}$$

(26)

The one-parameter exponential or Darmois-Koopman-Pitman families such as the Normal distribution arise naturally from the optimization of the Shannon entropy of $P(\chi)$ under normalization, first and second moment $\sigma_{\chi}^{2}=\frac{2}{3}m^{3}$ constraints [13]

	$\displaystyle L[p]$	$\displaystyle=$	$\displaystyle-\sum_{i}p_{i}\log p_{i}-\beta_{0}(\sum_{i}p_{i}-1)$
			$\displaystyle-\beta_{1}(\sum_{i}\alpha_{i}p_{i}-\mu_{\chi})-\beta_{2}(\sum_{i}\alpha_{i}^{2}p_{i}-\sigma_{\chi}^{2})$

optimization yields $p_{i}=\exp(1-\beta_{0}-\beta_{1}\alpha_{i}-\beta_{2}\alpha_{i}^{2})$. Using the normalization condition, together with $\mu_{\chi}=0$ we get

	$\displaystyle p_{i}$	$\displaystyle=$	$\displaystyle\frac{e^{-\beta\alpha_{i}^{2}}}{\sum_{j}e^{-\beta\alpha_{j}^{2}}}$
		$\displaystyle=$	$\displaystyle\frac{a_{i}}{\sum_{j}a_{j}}$

$$P_{\chi_{n}}(\alpha_{i})=\sqrt{\frac{3}{\pi}}m^{-\frac{3}{2}}\exp\left(-\frac{3\alpha_{i}^{2}}{4m^{3}}\right)$$

(30)

$$f_{\chi}(\alpha;\mu_{\chi},\sigma_{\chi})=\frac{1}{\sigma_{\chi}\sqrt{2\pi}}\exp\left(-\frac{(\alpha-\mu_{\chi})^{2}}{2\sigma_{\chi}^{2}}\right),$$

(31)

	$\displaystyle\bar{\alpha}$	$\displaystyle=$	$\displaystyle\frac{1}{N}\sum_{\tau\in T_{L}}\alpha(\tau)$		(32)
	$\displaystyle s_{\alpha}^{2}$	$\displaystyle=$	$\displaystyle\frac{1}{N-1}\sum_{\tau\in T_{L}}\alpha(\tau)^{2}$		(33)

As an illustration, let us apply our framework as explained at the beginning of Sec. III to white noise time series of length $L=10^{5}$ generated from different distributions: Standard Normal (unbounded support), Cauchy (unbounded support, heavy tails), Exponential (asymmetric distribution, unbounded support), Poisson (discrete unbounded support), Continuous Uniform (bounded support), and the special case of deterministic chaotic trajectories generated from the logistic map

$$f(x)=rx(1-x),\ \ x\in[0,1],\ \ r\in[0,4]$$

(34)

at control parameter value $r=4$. At this value of $r$, the logistic map is ergodic and its invariant density has a closed form $\rho(x)=1/\pi\sqrt{x(1-x)}$, which is equal to $\mathrm{Beta}(1/2,1/2)$. Furthermore, its orbits display exponential decay of correlations [14] and thus are effectively random in the long run. Therefore we can regard long time series obtained from 34 at $r=4$ as white noise generated from a Beta distribution as a source, i.e., $X\sim\mathrm{Beta}(1/2,1/2)$. As discussed in Sec. III, the validity of the theory developed for our functional requires the process $X_{t}$ to have a source distribution with continuous support, since a total order is needed for obtaining well-defined ranking permutations, but this is not the case for the Poisson distribution. Nevertheless, here we relax this condition to see that, in a practical situation where the discrete support of $X_{t}$ is large enough to make the probability of repeated neighbouring points in time very low, then we can expect our method to apply to a good approximation.

For the sake of comparison, we use the standardized variable $Z_{\alpha}$, whose associated process is $\{\alpha_{z,k}\}_{k\in[N]}$. Now let us choose an embedding dimension (sliding window length) with $n=32$ and let us use a lag of $l=1$. With this choice we ensure that the shape of $P_{\chi}(\alpha)$ is well approximated by a Gaussian (see Fig. 2-(d)).

In Fig 3, we can confirm empirically the statement that the only requirements for obtaining a Gaussian distribution for $\alpha(\tau)$, are the statistical independence in the observations and the continuity of the support of $X_{t}$. These conditions can be relaxed to include processes with sufficiently rapid (i.e. exponential) decay of correlations as in the case of the deterministic chaotic trajectory, or to discrete processes with sufficiently large support.

The usefulness of our analysis is not limited to large values of the embedding dimension. In Fig. 4 we show the distributions $P(Z_{\alpha})$ for embedding dimensions $n=2,4,8$, obtained from a chaotic trajectory of length $L=10^{4}$ from the logistic map with $r=4$ and initial condition $x_{0}=0.84291157\ldots$ . The discrepancies between the distributions for the chaotic trajectory and realizations of uniform white noise come from the impossibility of the logistic map of displaying some types of patterns, known as forbidden patterns [15]. For instance, in [15] it is shown that the pattern of the type $\langle 3,2,1\rangle$, and more generally patterns of the form $\langle*,3+k,*,2+k,*,1+k,*\rangle$ called outgrowth patterns (where $*$ indicates any other symbol in the pattern) cannot be displayed by orbits of the logistic map with $r=4$. Although one of the drawbacks of our method is that we are limited to even values of $n$, we can still see the effect of the forbidden patterns by the asymmetry in the distributions in Fig 4, which has less mass in the negative side of the support. This is because the forbidden pattern has a negative value of our functional $\alpha(321)=-2$, and thus, patterns that are negative in the $\alpha$ sense will be more likely to belong to the outgrowth set patterns of $\langle 3,2,1\rangle$. Correspondingly, an excess of positive patterns is observed, yielding an asymmetric distribution. This makes our analysis potentially useful for detecting signatures of determinism in an observed process by direct comparison with white noise. As it can be seen in Fig. 4, this effect is lost for a larger value of the lag due to the increased scale of observation and consequent loss of correlations.