This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

Solutions of the Multivariate Inverse Frobenius–Perron Problem

Colin Fox Department of Physics, University of Otago, Dunedin, New Zealand. email:[email protected]    Li-Jen Hsiao System Manufacturing Center, National Chung-Shan Institute of Science & Technology, Taiwan. email:[email protected]    Jeong Eun (Kate) Lee Department of Statistics, The University of Auckland, Auckland, New Zealand. email:[email protected]
Abstract

We address the inverse Frobenius–Perron problem: given a prescribed target distribution ρ\rho, find a deterministic map MM such that iterations of MM tend to ρ\rho in distribution. We show that all solutions may be written in terms of a factorization that combines the forward and inverse Rosenblatt transformations with a uniform map, that is, a map under which the uniform distribution on the dd-dimensional hypercube as invariant. Indeed, every solution is equivalent to the choice of a uniform map. We motivate this factorization via 11-dimensional examples, and then use the factorization to present solutions in 11 and 22 dimensions induced by a range of uniform maps.

I Introduction

A basic question in the theory of discrete dynamical systems, and in statistical mechanics, is whether a chaotic iterated function M:XXM:X\rightarrow X, that maps a space XdX\subseteq\mathbb{R}^{d} back onto XX, has an equilibrium distribution, with probability density function (PDF) ρ(x)\rho(x) 111The density function is with respect to some underlying measure, typically Lebesgue. Throughout this paper we will use the same symbol for the distribution and associated PDF, with meaning taken from context.. A necessary condition is that the distribution ρ\rho is invariant under MM, i.e. if xρx\sim\rho (xx is distributed as ρ\rho) then so is M(x)M(x), and further that the orbit of almost all points xXx\in X defined as 𝒪+(x)={x,M(x),M2(x),M3(x),}\mathcal{O}^{+}(x)=\left\{x,M(x),M^{2}(x),M^{3}(x),\ldots\right\} tends in distribution to ρ\rho. Then, under mild conditions, the map is ergodic for ρ\rho, that is, expectations with respect to ρ\rho may be replaced by averages over the orbit dorfman1999introduction ; LasotaMackey .

For example, it is well known UlamvonNeumann1947 ; may1976simple ; dorfman1999introduction ; LasotaMackey that the logistic map Mlog(x)=4x(1x)M_{\text{log}}(x)=4x(1-x), for x[0,1]x\in[0,1], is chaotic with the equilibrium distribution having PDF

ρlog(x)=1πx(1x),\rho_{\text{log}}(x)=\frac{1}{\pi\sqrt{x(1-x)}},

implying that MlogM_{\text{log}} is ergodic for ρlog\rho_{\text{log}}.

Our motivating interest is in using this ergodic property to implement sample-based inference for Bayesian analysis or machine learning. In those settings, the target distribution ρ\rho is defined by the application. Generating a sequence {x0,x1,x2,x3,}\left\{x_{0},x_{1},x_{2},x_{3},\ldots\right\} that is ergodic for ρ\rho is useful because then expectations of any quantity of interest can be computed as averages over the sequence, i.e.,

limN1Ni=0N1g(xi)=Xg(x)ρ(x)dx.\lim_{N\to\infty}\frac{1}{N}\sum_{i=0}^{N-1}g\left(x_{i}\right)=\int_{X}g(x)\rho(x)\,\mathrm{d}x. (1)

Commonly in statistics, such ergodic sequences are generated using a stochastic iteration that simulates a Markov chain targeting ρ\rho mcmc_liu ; here we explore deterministic iterations that generate an orbit that is ergodic for ρ\rho.

The equilibrium distribution for a given iterated function, if it exists, can be approximated by computing the orbit of the map for some starting point then performing kernel density estimation, or theoretically by seeking the stationary distribution of the Frobenius–Perron (FP) operator that is the transfer KlusKoltaiSchutte2016 , aka push-forward, operator induced by a deterministic map dorfman1999introduction ; LasotaMackey ; we present the FP equation in Section II.

The inverse problem that we consider here, of determining a map that gives a prescribed equilibrium distribution, is the inverse Frobenius–Perron problem (IFPP) and has been studied extensively grossmann1977invariant ; ershov1988solution ; diakonos1996construction ; diakonos1999stochastic ; pingel1999theory ; bollt2000controlling ; nie2013new ; nie2018matrix . Summaries of previous approaches to the IFPP are presented in bollt2000controlling ; rogers2008synthesis that characterize approaches as based on conjugate functions (see grossmann1977invariant for details) or the matrix method (see rogers2008synthesis for details), and wei2015solutions that also lists the differential equation approach. Existing work almost solely considers the IFPP in one-variable, d=1d=1, the exception being the development of a two-dimensional solution in bollt2000controlling that is also presented in rogers2008synthesis .

The matrix method, first suggested by Ulam Ulam1960 , solves the IFPP for a piecewise-constant approximation to the target density, using a transition matrix representation of the approximated FP operator. Convergence of the discrete approximation is related to Ulam’s conjecture, and has been proved for the multidimensional problem; see wei2015solutions and references therein. While the matrix method allows construction of solutions, at least in the limit, existing methods only offer a limited class of very non-smooth solutions, so are not clearly useful for characterizing all solutions, as we do here. We do not further consider the matrix methods.

The development in this paper starts with the differential equation approach in which the IFPP for restricted forms of distributions and maps is written as a differential equation that may be solved. We re-derive some existing solutions to the IFPP this way in Sec. III. The main contribution of this paper is to show that the form of these solutions may be generalized to give the general solution of the IFPP for any probability distribution in any dimension dd, as presented in the factorization theorem of Sec. IV. This novel factorization represents solutions of the IFPP in terms of the Rosenblatt transformation rosenblatt-1952 and a uniform map, that is a map on [0,1]d[0,1]^{d} that leaves the uniform distribution invariant. In particular, we show that the conjugating functions in grossmann1977invariant are exactly the inverse Rosenblatt transformations. For a given Rosenblatt transformation, there is a one-to-one correspondence between solution of the IFPP and choice of a uniform map.

This reformulation of the IFPP in terms of two well-studied constructs leads to practical analytic and numerical solutions by exploiting existing, well-developed methods for Rosenblatt transformations, and for deterministic iterations that target the uniform distribution. The factorization also allows us to establish equivalence of solutions of the IFPP and other methods that employ a deterministic map within the generation of ergodic sequences. This standardizes and simplifies existing solution methods by showing that they are special cases of constructing the Rosenblatt transformation (or its inverse), and selection of a uniform map.

This paper starts with definitions of the IFPP and the Lyuapunov exponent in Sec. II. Solutions of the IFPP in one-dimension, d=1d=1, are developed in Sec. III. These solutions for d=1d=1 motivate the factorization theorem in Sec. IV that presents a general solution to the IFPP for probability distributions with domains in d\mathbb{R}^{d} for any dd. Sec. V presents further examples of univariate, d=1d=1, solutions to the IFPP based on the factorization theorem in Sec. IV. Two 2-dimensional numerical examples are presented in Section VI.3 to demonstrate that the theoretical constructs may be implemented in practice. A summary and discussion of results is presented in Section VII, including a discussion of some existing computational methods that can be viewed as implicitly implementing the factorization solution of the IFPP presented here.

II Inverse Frobenius–Perron Problem and Lyapunov Exponent

In this section we define the forward and inverse Frobenius–Perron problems, and also the Lyuapunov exponent that measures chaotic behaviour.

II.1 Frobenius–Perron Operator

A deterministic map xn+1=M(xn)x_{n+1}=M(x_{n}) defines a map on probability distributions over state, called the transfer operator KlusKoltaiSchutte2016 . Consider the case where the initial state x0ρ0()x_{0}\sim\rho_{0}(\cdot) (x0x_{0} is distributed as ρ0\rho_{0}) for some distribution ρ0\rho_{0} and let ρn\rho_{n} denote the nn-step distribution, i.e., the distribution over xn=Mn(x0)x_{n}=M^{n}(x_{0}) at iteration nn. The transfer operator that maps ρnρn+1\rho_{n}\mapsto\rho_{n+1} induced by MM is given by the Frobenius–Perron operator associated with M:xyM:x\mapsto y dorfman1999introduction ; LasotaMackey ; Gaspard1998

ρn+1(y)=xM1(y)ρn(x)|J(x)|\rho_{n+1}(y)=\sum_{x\in M^{-1}(y)}\frac{\rho_{n}\left(x\right)}{|J(x)|} (2)

where |J(x)||J(x)| denotes the Jacobian determinant 222We have used the language of differential maps, as all the maps that we display in this paper are differentiable almost everywhere varberg1971change. More generally, |J(x)|1|J(x)|^{-1} denotes the density of ρnM1\rho_{n}M^{-1} with respect to ρn+1\rho_{n+1}, see, e.g., (LasotaMackey, , Remark 3.2.4.). of MM at xx, and the sum is over inverse images of yy.

The equilibrium distribution ρ\rho of MM satisfies

ρ(y)=xM1(y)ρ(x)|J(x)|\rho(y)=\sum_{x\in M^{-1}(y)}\frac{\rho(x)}{|J(x)|} (3)

and we say that ρ\rho is invariant under MM.

II.2 Inverse Frobenius–Perron Problem

The inverse problem that we address is finding an iterative map MM that has a given distribution ρ\rho as its equilibrium distribution. We do this by performing the inverse Frobenius–Perron problem (IFPP) of finding MM that satisfies (3) to ensure that ρ\rho is an invariant distribution of MM. Establishing chaotic hence ergodic behaviour is a separate calculation.

We will assume throughout that ρ\rho is absolutely continuous with respect to the underlying measure so that a probability density function ρ(x)\rho(x) exists, and, furthermore, that ρ(x)>0\rho(x)>0, xX\forall x\in X.

II.3 Lyapunov Exponent

The Lyapunov exponent hh of an iterative map gives the average exponential rate of divergence of trajectories. We define the (maximal) Lyapunov exponent hh as dorfman1999introduction ; LasotaMackey

h=limN1Nlog|dxNdx0|=limN1Nn=0N1log|J(xn)|h=\lim_{N\to\infty}\frac{1}{N}\log\left|\frac{\mathrm{d}x_{N}}{\mathrm{d}x_{0}}\right|=\lim_{N\to\infty}\frac{1}{N}\sum^{N-1}_{n=0}\log|J(x_{n})| (4)

that features the starting value x0x_{0}. For ergodic maps the dependency on x0x_{0} is lost as NN\to\infty and the Lyapunov exponent may be written

h=Xlog|J(x)|ρ(x)dxh=\int_{X}\log|J(x)|\rho(x)\,\mathrm{d}x (5)

where ρ(x)\rho(x) is the invariant density. A positive Lyapunov exponent hh indicates that the map is chaotic.

The theoretical value for the Lyapunov exponent may be obtained using (5), while (4) provides an empirical value obtained by iterating the map MM. For example, the Lyapunov exponent of the logistic map evaluated by (5) is hlog=log20.693147h_{\text{log}}=\log 2\approx 0.693147 while (4) evaluated over an orbit with 1000010000 iterations gave hlog0.693140h_{\text{log}}\approx 0.693140.

For chaotic maps, any uncertainty in initial value means that an orbit cannot be precisely predicted, since initial states with any separation become arbitrarily far apart, within XX, as iterations increase. Hence it is useful to characterize the orbit statistically, in terms of the equilibrium distribution over states in the orbit.

It is interesting to note that theoretical chaotic and ergodic behaviour does not necessarily occur when iterations of a map are implemented on a finite-precision computer. For example, when the logistic map, above, on [0,1][0,1] is iterated on a binary computer the multiplication by 44 corresponds to a shift left by two bits and all subsequent operations maintain lowest order bits that are zero. Repeated iterations eventually produce the number zero, no matter the starting value. While it is simple to correct this non-ideal behaviour, as was done to give the numerical Lyapunov exponent, above, it is important to note that computer implementation can have very different dynamics to the mathematical model.

III Solution of the IFPP in 11-dimension

In this Section we develop solutions of the IFPP in one-dimension, d=1d=1. Without loss of generality we consider distributions on the unit interval X=[0,1]X=[0,1], as the domain of any univariate distribution may be transformed by a change of variable to X=[0,1]X=[0,1], including when the domain is the whole real line (,)(-\infty,\infty).

For distributions over a scalar random variable, the FP equation for the invariant density (3) simplifies to

ρ(y)=xM1(y)ρ(x)|M(x)|.\rho(y)=\sum_{x\in M^{-1}(y)}\frac{\rho\left(x\right)}{|M^{\prime}(x)|}. (6)

III.1 The Simplest Solution

We first note, almost trivially, that the identity map M=IM=I, where I(x)=xI(x)=x, has ρ\rho as an invariant distribution, so solves the IFPP for any ρ\rho. Somewhat less trivial is to derive this simplest solution by assuming that MM is monotonic increasing and M(0)=0M(0)=0, so that there is just one inverse image in (6). Writing |M(x)|=dM/dx|M^{\prime}(x)|=\mathrm{d}M/\mathrm{d}x gives the differential equation with separated variables

ρ(M)dM=ρ(x)dx\rho(M)\mathrm{d}M=\rho(x)\mathrm{d}x (7)

that has solution

F(M)=F(x)F(M)=F(x)

where F(x)=0xρ(x)dxF(x)=\int_{0}^{x}\rho(x^{\prime})\,\mathrm{d}x^{\prime} is the cumulative distribution function (CDF) for ρ\rho. If FF is invertible denote the inverse by F1F^{-1}, called the the inverse distribution function (IDF), otherwise let F1F^{-1} denote the generalized inverse distribution function, F1(p)=inf{xX:F(x)p}\displaystyle F^{-1}(p)=\inf\{x\in X:F(x)\geq p\}. Then, M=F1(F(x))=xM=F^{-1}(F(x))=x, or M(x)=xM(x)=x, almost everywhere. Hence the identity map is the unique monotonic increasing map that has ρ\rho as its invariant distribution. Clearly, the identity map is not ergodic for ρ\rho.

We may generalize this solution by setting M(0)=kM(0)=k, for some k[0,1)k\in[0,1), and also only requiring MM to be piecewise continuous. Allowing one discontinuity in MM we write the integral of the separated differential equation (7) as

F(M)=F(x)+kmod1F(M)=F(x)+k\mod 1

giving the solution to the IFPP

M(x)=(F1TcF)(x)M(x)=\left(F^{-1}\circ T^{c}\circ F\right)(x) (8)

where c=F1(k)c=F^{-1}(k). Here TcT^{c} denotes the operator that translates by cc with wrap-around on [0,1)[0,1) 333Hence TcT^{c} is the translation operator on the unit circle S1S^{1}

Tc(y)=y+cy+c,T^{c}(y)=y+c-\lfloor y+c\rfloor, (9)

where \displaystyle\lfloor\,\ \rfloor denotes the floor function; see Figure 1 (left). The identity map is recovered when k=c=0k=c=0.

Refer to caption
Refer to caption
Figure 1: Translation operator TcT^{c} for c=0.3c=0.3 (left) and triangle map t1t_{1} (right).

It is easily seen that the Lyapunov exponent of the map in (8) is h=0h=0, so the map is not chaotic. However, interestingly, this map can generate a sequence of states that produce a numerical integration rule with respect to ρ\rho, since appropriate choices of cc and number of iterations NN can produce a rectangle rule quadrature or a quasi Monte Carlo lattice rule; see Sec. IV.3.

III.2 Exploiting Symmetry in ρ(x)\rho(x)

When the PDF ρ\rho has reflexive symmetry about 1/21/2, i.e. ρ(x)=ρ(1x)\rho(x)=\rho(1-x), we can simplify the FP equation (6) by assuming that the map MM has the same symmetry. Specifically, we write the triangle map (see Figure 1 (right))

t1(x)=12|x1/2|t_{1}(x)=1-2\left|x-1/2\right| (10)

that has reflexive symmetry about 1/21/2, and write

M(x)=m(t1(x))M(x)=m(t_{1}(x)) (11)

where m(x):[0,1][0,1]m(x):[0,1]\to[0,1] is a monotonic increasing map with m(0)=0m(0)=0 (and, it will turn out, m(1)=1m(1)=1). Hence the FP equation simplifies to

ρ(y)=2ρ(x)|M(x)|,xM1(y),\rho(y)=2\frac{\rho\left(x\right)}{|M^{\prime}(x)|},\quad x\in M^{-1}(y), (12)

that we can write as the separated equations

ρ(M)dM=2ρ(x)dx,\displaystyle\rho(M)\,\mathrm{d}M=2\rho(x)\mathrm{d}x,\quad x<1/2,\displaystyle x<1/2,
ρ(M)dM=2ρ(x)dx,\displaystyle\rho(M)\,\mathrm{d}M=-2\rho(x)\mathrm{d}x,\quad x>1/2,\displaystyle x>1/2,

that has the continuous solution

F(M)=t1(F(x))F(M)=t_{1}(F(x))

giving the continuous solution to the IFPP

M(x)=(F1t1F)(x).M(x)=\left(F^{-1}\circ t_{1}\circ F\right)(x). (13)

One can solve for m(x)=F1(2F(x/2))m(x)=F^{-1}(2F(x/2)), though we do not further consider the function mm.

The approach we have used here simplifies the approach in diakonos1996construction , while ‘doubly symmetric’ maps of the form (11) were considered in gyorgyi1984fully , and again in pingel1999theory ; diakonos1999stochastic .

III.3 Symmetric Triangular Distribution

To give a concrete example of the solution in (13), we consider the symmetric triangular distribution on [0,1][0,1] with PDF

ρtri(x)=24|x12|\rho_{\text{tri}}(x)=2-4\left|x-\frac{1}{2}\right| (14)

that has reflexive symmetry about x=12x=\frac{1}{2}. The CDF is

Ftri(x)={2x20x1212+4x2x212x1F_{\text{tri}}(x)=\begin{cases}2x^{2}&0\leq x\leq\frac{1}{2}\\ \frac{1}{2}+4x-2x^{2}&\frac{1}{2}\leq x\leq 1\end{cases}

giving the unimodal map, after substituting into (13),

Mtri(x)={2x0x181122x218x121122(1x)212x1182(1x)118x1M_{\text{tri}}(x)=\left\{\begin{array}[]{rl}\sqrt{2}x&0\leq x\leq\frac{1}{\sqrt{8}}\\ 1-\sqrt{\frac{1}{2}-2x^{2}}&\frac{1}{\sqrt{8}}\leq x\leq\frac{1}{2}\\ 1-\sqrt{\frac{1}{2}-2(1-x)^{2}}&\frac{1}{2}\leq x\leq 1-\frac{1}{\sqrt{8}}\\ \sqrt{2}(1-x)&1-\frac{1}{\sqrt{8}}\leq x\leq 1\end{array}\right. (15)

shown in Figure 2 (left). The same map was derived in grossmann1977invariant . Figure 2 (right) shows a normalized histogram of 10610^{6} iterates of MtriM_{\text{tri}} starting at x=0.3x=0.3, confirming that the orbit of MtriM_{\text{tri}} converges to the desired triangular distribution. The numerical implementation avoids finite-precision effects, as discussed later.

Refer to caption
Refer to caption
Figure 2: Iterative map MtriM_{\text{tri}} in (15) (left) and a histogram of 1×1061\times 10^{6} iterates of the map MtriM_{\text{tri}} (right).

The theoretical Lyapunov exponent for MtriM_{\text{tri}} is htri=log20.693147h_{\text{tri}}=\log 2\approx 0.693147 while (4) evaluated over an orbit with 10610^{6} iterations gave htri0.693148h_{\text{tri}}\approx 0.693148.

IV Solutions of the IFPP for General Multi-Variate Target Distributions

The solutions to the 11-dimensional IFPP with special structure in Eqs (8) and (13) are actually examples of a general solution to the IFPP for multi-variate probability distributions with no special structure. We state that connection via a theorem that establishes a factorization of all possible solutions to the IFPP, and that also provides a practical means of solving the IFPP.

We first introduce the forward and inverse Rosenblatt transformations, that is the multi-variate generalization the CDF and IDF for univariate distributions.

IV.1 Forward and Inverse Rosenblatt Transformations

A simple transformation of an absolutely continuous dd-variate distribution into the uniform distribution on the dd-dimensional hypercube was introduced by Rosenblatt rosenblatt-1952 , as follows. The joint PDF can be written as a product of conditional densities,

ρ(x1,,xd)=ρ1(x1)ρ2(x2|x1)ρd(xd|x1,xd1),\rho(x_{1},\ldots,x_{d})=\rho_{1}(x_{1})\rho_{2}(x_{2}|x_{1})\cdots\rho_{d}(x_{d}|x_{1}\ldots,x_{d-1}),

where ρk(xk|x1,xk1)\rho_{k}(x_{k}|x_{1}\ldots,x_{k-1}) is a conditional density given by

ρk(xk|x1,xk1)=pk(x1,,xk)pk1(x1,xk1),\rho_{k}(x_{k}|x_{1}\ldots,x_{k-1})=\frac{p_{k}(x_{1},\ldots,x_{k})}{p_{k-1}(x_{1}\ldots,x_{k-1})}, (16)

in terms of the marginal densities,

pk=ρ(x1,,xd)dxk+1dxd,p_{k}=\int\rho(x_{1},\ldots,x_{d})\,\mathrm{d}x_{k+1}\cdots\mathrm{d}x_{d}, (17)

where k=1,,dk=1,\ldots,d.

Let z=(z1,,zd)=R(x1,,xd)z=(z_{1},\ldots,z_{d})=R(x_{1},\ldots,x_{d}) where RR is the Rosenblatt transformation rosenblatt-1952 from the state-space XdX\subseteq\mathbb{R}^{d} of ρ\rho to the dd-dimensional unit cube [0,1]d[0,1]^{d}, defined in terms of the (cumulative) distribution function FF by

z1\displaystyle z_{1} =F1(x1)=x1ρ1(x1)dx1,\displaystyle=F_{1}(x_{1})=\int_{-\infty}^{x_{1}}\rho_{1}(x_{1}^{\prime})\,\mathrm{d}x_{1}^{\prime},
z2\displaystyle z_{2} =F2(x2|x1)=x2ρ2(x2|x1)dx2,\displaystyle=F_{2}(x_{2}|x_{1})=\int_{-\infty}^{x_{2}}\rho_{2}(x_{2}^{\prime}|x_{1})\,\mathrm{d}x_{2}^{\prime},
\displaystyle\vdots
zd\displaystyle z_{d} =Fd(x2|x1,,xd1)=xdρd(xd|x1,,xd1)dxd.\displaystyle=F_{d}(x_{2}|x_{1},\ldots,x_{d-1})=\int_{-\infty}^{x_{d}}\rho_{d}(x_{d}^{\prime}|x_{1},\ldots,x_{d-1})\,\mathrm{d}x_{d}^{\prime}.

As noted in rosenblatt-1952 , there are d!d! transformations of this type, corresponding to the d!d! ways of ordering the coordinates. Further multiplicity is introduced by considering coordinate transformations, such as rotations.

Notice that in 11-dimension, the Rosenblatt transformation R(x)R(x) is simply the CDF F(x)F(x).

It follows that if xρx\sim\rho then z=R(x)Unif([0,1]d)z=R(x)\sim\operatorname{Unif}([0,1]^{d}), i.e., zz is uniformly distributed on the dd-dimensional unit cube rosenblatt-1952 . When ρ(x)>0\rho(x)>0, xX\forall x\in X the distribution functions are strictly monotonic increasing and the inverse of the Rosenblatt transformation R1R^{-1} is well defined, otherwise let R1R^{-1} denote the generalized inverse as in Sec. III.1. Then, if zUnif([0,1]d)z\sim\operatorname{Unif}([0,1]^{d}) it follows that x=R1(z)ρx=R^{-1}(z)\sim\rho, i.e., xx is distributed as the desired target distribution ρ\rho johnson1987multivariate ; this is the basis of the conditional distribution method for generating multi-variate random variables, that generalizes the inverse cumulative transformation method for univariate distributions devroye-rvgen-1986 ; johnson1987multivariate ; hormann-rvgen-2004 ; dolgov2020approximation . These results may also be established by substituting RR or R1R^{-1} into the the FP equation (2), noting that there is a single inverse image and that the Jacobian determinant of RR equals the target PDF ρ(x)\rho(x).

In the remainder of this paper, we refer to any map RR satisfying xρR(x)Unif([0,1]d)x\sim\rho\Rightarrow R(x)\sim\operatorname{Unif}([0,1]^{d}) as a Rosenblatt transformation, with (generalized) inverse as defined above.

IV.2 Factorization Theorem

The following theorem characterizes solutions to the IFPP.

Theorem 1.

Given a probability distribution ρ\rho in dd-dimensions, a map M(x)M(x) is a solution of the IFPP, that is, M(x)M(x) satisfies the FP equation (3), if and only if

M(x)=(R1UR)(x),M(x)=(R^{-1}\circ U\circ R)(x), (18)

where RR is a Rosenblatt transformation and UU is a ‘uniform map’ on the unit dd-dimensional hypercube, i.e. a map that has Unif([0,1]d)\operatorname{Unif}([0,1]^{d}) as invariant distribution 444When UU satisfies the stronger condition that Unif([0,1]d)\operatorname{Unif}([0,1]^{d}) is the equilibrium distribution, UU is called an exact map LasotaMackey ..

Proof: We will show that ρ\rho is an invariant distribution of MM iff MM has the form (18). (\Rightarrow) Assume MM has the form (18). If xρx\sim\rho then R(x)Unif([0,1]d)R(x)\sim\operatorname{Unif}([0,1]^{d}) hence U(R(x))Unif([0,1]d)U(R(x))\sim\operatorname{Unif}([0,1]^{d}), as Unif([0,1]d)\operatorname{Unif}([0,1]^{d}) in invariant under UU, hence M(x)=R1(U(R(x)))ρM(x)=R^{-1}(U(R(x)))\sim\rho, as desired. (\Leftarrow) If ρ\rho is invariant under MM then U=RMR1U=R\circ M\circ R^{-1} is a uniform map, since if zUnif([0,1]d)z\sim\operatorname{Unif}([0,1]^{d}) then R1(z)ρR^{-1}(z)\sim\rho, M(R1(z))ρM(R^{-1}(z))\sim\rho, and R(M(R1(z)))Unif([0,1]d)R(M(R^{-1}(z)))\sim\operatorname{Unif}([0,1]^{d}). Inserting this UU into (18) gives the desired factorization.  ∎

The first part of the proof shows that any uniform map UU induces a solution to the IFPP, though the particular solution depends on the particular Rosenblatt transformation. The second part of the proof shows that different solutions to the IFPP effectively differ only by the choice of the uniform map UU, once the Rosenblatt transformation is determined, that is, a coordinate system is chosen with an ordering of those coordinates.

Grossmann and Thomae grossmann1977invariant called dynamical systems MM and UU related by a formula of the form (18) as ‘related by conjugation’, or just ‘conjugate’, and the map R1R^{-1} in (18) is a ‘conjugating function’. Thus, in the language of grossmann1977invariant , Theorem 1 shows that the IFPP for any distribution ρ\rho has a solution (actually, it shows that there are infinitely many solutions), every solution map is conjugate to a uniform map, and the conjugating function is precisely the inverse Rosenblatt transformation.

Notice that both the translation operator TcT^{c} in (9) and the triangle map in (10) are uniform maps on the unit interval [0,1][0,1]. Thus, the solutions to the IFPP given in Eqs (8) and (13) are examples of the general solution form in (18). In particular, while the solution to the IFPP in (13) was derived assuming symmetry of the target density ρ()\rho(\cdot), (13) actually gives a solution of the IFPP for any density ρ()\rho(\cdot). Unimodal maps of this form were derived in ershov1988solution .

Computed examples of solutions to the IFPP given by the factorization (18) are presented in Sec. V, in one dimension, and in Sec VI in two dimensions. High dimensional calculations are discussed in Sec. VII.

IV.3 Properties of MM from UU

Many properties of the map MM are inherited from the uniform map UU.

When RR and R1R^{-1} are continuous, MM is continuous iff UU is continuous. In one dimension, monotonicity of the CDF and IDF imply that the number of modes of UU equals the number of modes of MM; in particular, MM is unimodal iff UU is unimodal.

Constructing iterative maps with specific periodicity of the orbit is possible through the use of translation operators TcT^{c} as uniform maps, defined in Eq. (9). Consider, first, maps in one dimension on [0,1][0,1]. If the shift c0c\neq 0 and cc\notin\mathbb{Q}, the map is aperiodic. However, in the case that c0c\neq 0 and cc\in\mathbb{Q} such that

c=NDc=\frac{N}{D} (19)

with N,DN,D\in\mathbb{N} and gcd(N,D)=1\gcd(N,D)=1, then the map is periodic with periodicity DD, and iterative maps constructed with U=TcU=T^{c} exhibit the same periodicity. These properties may be extended to multi-dimensional settings when the translation constant cc is a vector of shifts in each coordinate direction, as used in rank-one lattice rules for quasi-Monte Carlo integration dick2013high .

The factorization in Theorem 1 also shows that performing an iteration xn+1=M(xn)x_{n+1}=M(x_{n}) with an iterative map MM on the space XX is equivalent to applying the corresponding uniform map zn+1=U(zn)z_{n+1}=U(z_{n}) on the space [0,1]d[0,1]^{d} through the transformations RR and R1R^{-1}, as indicated in the following (commuting) diagram.

xn{x_{n}}zn{z_{n}}xn+1{x_{n+1}}zn+1{z_{n+1}}R\scriptstyle{\displaystyle R}M\scriptstyle{\displaystyle M}U\scriptstyle{\displaystyle U}R1\scriptstyle{\displaystyle R^{-1}}R\scriptstyle{\displaystyle R}R1\scriptstyle{\displaystyle R^{-1}}

Using this commuting property, it is straightforward to prove the following lemma.

Lemma 1.

For given distribution ρ\rho, let RR be a Rosenblatt transformation for ρ\rho. Let M=R1URM=R^{-1}\circ U\circ R be a solution of the IFFP, as guaranteed by Theorem 1, where UU is a uniform map. Then

𝒪M+(x0)=R1(𝒪U+(R(x0))).\mathcal{O}^{+}_{M}(x_{0})=R^{-1}\left(\mathcal{O}^{+}_{U}(R(x_{0}))\right). (20)

Hence, instead of iterating MM on the space XX to produce the sequence {x1,x2,x3,}\{x_{1},x_{2},x_{3},\ldots\}, Eq. (20) shows that one can iterate the map UU on the space [0,1]d[0,1]^{d} to produce the sequence {z1,z2,z3,}\{z_{1},z_{2},z_{3},\ldots\} and then evaluate xn=R1znx_{n}=R^{-1}z_{n}, n=1,2,n=1,2,\ldots, to produce exactly the same sequence on XX. Since the map MM is mixing or ergodic iff the uniform map UU is mixing or ergodic, respectively, in this sense mixing and ergodicity of MM is inherited from UU.

Using the expansion in Eq. (20), we see that MM is deterministic or stochastic iff UU is deterministic or stochastic, respectively. Even though we are not considering stochastic maps here, we note that, for stochastic maps, iterations of MM are correlated or independent iff iterations of UU are correlated or independent, respectively.

Some other properties that are, and are not, preserved by the transformation from UU to MM are discussed in grossmann1977invariant .

V Examples in One Dimension

V.1 Uniform Maps on [0,1][0,1]

We have already encountered three uniform maps on the interval [0,1][0,1], namely the identity map I(x)=xI(x)=x (Figure 3 (top, left)) and the translation operator (9) (Figure 1 (left)), that have Lyapunov exponent h=0h=0, and the triangle map (Figure 1 (right)) with Lyapunov exponent h=log2h=\log 2.

Some further elementary uniform maps on [0,1][0,1], and associated Lyapunov exponents, are:

  • \ell periods of a sawtooth function on [0,1][0,1] (Figure 3, top-right, for l=3l=3 periods) 555The two period sawtooth map s2s_{2} is also called the Bernoulli map, and its orbit 𝒪+(x)\mathcal{O}^{+}(x) is the dyadic transformation.

    s(x)=xx,s_{\ell}(x)=\ell*x-\lfloor\ell*x\rfloor, (21)

    with Lyapunov exponent h=logh=\log\ell,

  • \ell periods of a triangle function on [0,1][0,1] (Figure 3 (bottom, left) for l=3l=3 periods) 666This is the ‘broken linear transformation’ in grossmann1977invariant of order p=2p=2\ell.

    t(x)=12|s(x)1/2|.t_{\ell}(x)=1-2\left|s_{\ell}(x)-1/2\right|. (22)

    with Lyapunov exponent h=log2h=\log 2\ell,

  • and the asymmetric triangle, for c(0,1)c\in(0,1) (Figure 3 (bottom, right) for c=0.3c=0.3)

    tc(x)={xc0xc1x1ccx1t^{c}(x)=\begin{cases}\displaystyle\frac{x}{c}&0\leq x\leq c\\[10.00002pt] \displaystyle\frac{1-x}{1-c}&c\leq x\leq 1\end{cases} (23)

    with Lyapunov exponent 0h=clogc(1c)log(1c)log20\leq h=-c\log c-(1-c)\log(1-c)\leq\log 2.

Refer to caption
Refer to caption
Refer to caption
Refer to caption
Figure 3: Four examples of uniform maps on [0,1][0,1]. Top-left: identity map. Top-right: \ell periods of a sawtooth wave for =3\ell=3. Bottom-left: \ell periods of a triangle wave for =3\ell=3. Bottom-right: asymmetric triangle for c=0.3c=0.3.

Obviously, many more uniform maps are possible. Further examples can be formed by partitioning the domain and range of any uniform map, and then permuting the subintervals. Many existing ‘matrix-based’ methods for constructing solutions to the IFPP, can be viewed as examples of such a partition-and-permute of an elementary uniform map rogers2004synthesizing . Uniform maps of other forms are developed in huang2005characterizing from two-segmental Lebesgue processes, producing uniform maps that are curiously non-linear.

Lemma 2.

The composition of uniform maps is also a uniform map, i.e., if U1U_{1} and U2U_{2} are uniform maps then so is U=U1U2U=U_{1}\circ U_{2}.

An example is tt_{\ell}, that can be constructed as the composition t=st1t_{\ell}=s_{\ell}\circ t_{1}.

We mentioned the numerical artifacts that can occur with finite-precision arithmetic, particularly when implementing maps on a binary computer and when the endpoints of the interval XX and constants in the maps have exact binary representations. We avoided these artifacts by composing the stated uniform map with the translation TcT^{c} for c=1/3×109c=1/3\times 10^{-9} that does not have finite binary representation 777Computation was performed in MatLab that implements IEEE Standard 754 for double-precision binary floating-point format.. This small shift is indiscernible in the graphs of the maps.

V.2 Ramp Distribution

To give a concrete example of the solution in (13) for a distribution without reflexive symmetry, we consider the ramp distribution with PDF

ρramp(x)=2x\rho_{\text{ramp}}(x)=2x (24)

that has CDF

Framp(x)=x2.F_{\text{ramp}}(x)=x^{2}.

We produce a unimodal, continuous map by choosing the uniform map t1t_{1}, as in (13), to give

Mramp={2x0x1221x212x1M_{\text{ramp}}=\begin{cases}\sqrt{2}x&0\leq x\leq\frac{1}{\sqrt{2}}\\ \sqrt{2}\sqrt{1-x^{2}}&\frac{1}{\sqrt{2}}\leq x\leq 1\end{cases} (25)

shown in Figure 4 (left). Figure 2 (right) shows a normalized histogram of 10610^{6} iterates of MrampM_{\text{ramp}} starting at x=0.3x=0.3, confirming that the orbit of MrampM_{\text{ramp}} converges to the desired ramp distribution, as guaranteed by Theorem 1. The numerical implementation avoids finite-precision effects, as discussed earlier.

Refer to caption
Refer to caption
Figure 4: Iterative map MrampM_{\text{ramp}} in (25) (left) and a normalized histogram of 1×1061\times 10^{6} iterates that approximates the equilibrium PDF (right).

The estimated Lyapunov exponent for this map is h1.040035h\approx 1.040035, which is greater than the Lyapunov exponent for the inducing triangular map t1t_{1} which is log20.693147\log 2\approx 0.693147.

Using a different uniform map gives a different solution to the IFPP. For example, choosing s3s_{3} gives the map M=F1s3FM=F^{-1}\circ s_{3}\circ F shown in Figure 5 (left). A normalized histogram over an orbit of 10610^{6} iterations is shown in Figure 5 (right), confirming that this map is also ergodic for ρramp\rho_{\text{ramp}}.

Refer to caption
Refer to caption
Figure 5: Iterative map MrampM_{\text{ramp}} in (25) (left) and a normalized histogram of 1×1061\times 10^{6} iterates that approximates the equilibrium PDF (right).

The estimated Lyapunov exponent for this map is h1.098612h\approx 1.098612, which is the same numerical value as the Lyapunov exponent for the sawtooth map s3s_{3} which is log31.098612\log 3\approx 1.098612. Comparing with the result for the map induced by t1t_{1}, this shows that the Lyapunov exponent of a map is not generally equal to the Lyapunov exponent of the inducing uniform map.

V.3 The Logistic Map and Alternatives

The logistic map, mentioned in the introduction, is

Mlog(x)=4x(1x).M_{\text{log}}(x)=4x(1-x). (26)

The equilibrium distribution of this map

ρlog(x)=1πx(1x),\rho_{\text{log}}(x)=\frac{1}{\pi\sqrt{x(1-x)}}, (27)

can be easily verified by substituting into the FP equation (6). The CDF of ρlog(x)\rho_{\text{log}}(x) is

Flog(x)=0xρlog(x)𝑑x=2πarcsin(x),F_{\text{log}}(x)=\int_{0}^{x}\rho_{\text{log}}(x^{\prime})dx^{\prime}=\frac{2}{\pi}\arcsin(\sqrt{x}), (28)

and the IDF is

Flog1(x)=sin2(πx2)=12(1cos(πx)).F_{\text{log}}^{-1}(x)=\sin^{2}(\frac{\pi x}{2})=\frac{1}{2}(1-\cos(\pi x)). (29)

The logistic map (26) is induced by the factorization (18) by choosing the triangle map t1t_{1} as uniform map, i.e., substituting the CDF (28) and IDF (29) into (13); see Figure 6 (left). Equivalently, one may note that the logistic map (26) is transformed into the triangle map t1t_{1} by the change of variables z=F1(x)z=F^{-1}(x); in the language of grossmann1977invariant , MlogM_{\text{log}} and t1t_{1} are conjugate dynamical laws.

Other iterative maps which preserves the same equilibrium distribution (27) can be constructed by choosing another uniform map, such as \ell periods of a triangle function (22). This gives the iterative maps

M=Flog1tFlog=sin2(2arcsin(x)),1,M_{\ell}=F_{\text{log}}^{-1}\circ t_{\ell}\circ F_{\text{log}}=\sin^{2}(2\ell\arcsin(\sqrt{x})),\ell\geq 1, (30)

that coincide with the nthn^{\text{th}} power of the logistic map (26) for =2n1\ell=2^{n-1}. Figure 6 (right) shows the map which preserves the same equilibrium distribution as the logistic map but induced by the uniform map t3t_{3}. Since 33 is not of the form 2n12^{n-1}, this map is not simply a power of the logistic map.

Refer to caption
Refer to caption
Figure 6: Logistic map, that equals Mlog=Flog1t1FlogM_{\text{log}}=F_{\text{log}}^{-1}\circ t_{1}\circ F_{\text{log}} (left), and the map M=Flog1t3FlogM=F_{\text{log}}^{-1}\circ t_{3}\circ F_{\text{log}} that has the same equilibrium distribution (right).

The theoretical value of the Lyapunov exponent of the map in (30) is log2\log 2\ell, using (5). Table 1 gives the theoretical values of the Lyapunov exponent and experimentally calculated values using 1000010000 iterations, as in (4), for some values of \ell.

\ell heh_{\text{e}} hth_{\text{t}}
11 0.6928190.692819 0.6931470.693147
22 1.3862841.386284 1.3862941.386294
44 2.0794302.079430 2.0794422.079442
2242^{24} 17.32859417.328594 17.32868017.328680
2392^{39} 27.72571327.725713 27.72588727.725887
Table 1: Experimental heh_{\text{e}} and theoretical hth_{\text{t}} Lyapunov exponents for the maps in (6) for a range of \ell, given to 66 decimal places.

VI Two Examples in Two Dimensions

VI.1 Uniform Maps on [0,1]2[0,1]^{2}

Two well-known examples of uniform maps in the two-dimensional unit square, X=[0,1]2X=[0,1]^{2}, are the baker’s map

Ub(x1,x2)=(2x1mod1,x2+u(x112)2),U_{\text{b}}(x_{1},x_{2})=\left(2x_{1}\mod 1,\frac{x_{2}+u\left(x_{1}-\frac{1}{2}\right)}{2}\right), (31)

where uu is the unit step function, and the Arnold cat map

UA(x1,x2)=((2x1+x2)mod1,(x1+x2)mod1).U_{\text{A}}(x_{1},x_{2})=\left((2x_{1}+x_{2})\mod 1,(x_{1}+x_{2})\mod 1\right). (32)

Other uniform maps in d>1d>1 dimensions may be formed by 11-dimensional uniform maps acting on each coordinate, giving the coordinate-wise uniform map

U(x)=(U1(x1),U2(x2),,Un(xd))U(x)=\left(U_{1}(x_{1}),U_{2}(x_{2}),\ldots,U_{n}(x_{d})\right) (33)

where Ui(x)U_{i}(x), i=1,2,,di=1,2,\ldots,d, are uniform maps in one dimension. We will use the baker’s map (31) and a coordinate-wise uniform map in the 22-dimensional examples that follow.

VI.2 Checker-Board Distribution

This example demonstrates construction of a map in two-dimensions that targets the checker board distribution, shown in Figure 8 (bottom-left) and Figure 9 (bottom-left), using the factorization (18).

The first step in constructing a solution to the IFPP for this distribution is to construct the forward and inverse Rosenblatt transformations, which requires the marginal density functions (17) that may be evaluated analytically in this case. A plot of the two components of the functions RR and R1R^{-1} is shown in Figure 7.

Refer to caption
Figure 7: Plots of the Rosenblatt transformation RR and its inverse R1R^{-1} for the checker-board distribution. Top row shows the components of RR: first component (left) and second component (right). Bottom row shows the components of R1R^{-1}: first component (left) and second component (right).

We construct two solutions to the IFPP, each induced by choosing a particular uniform map: the first is the baker’s map (31); the second is a component-wise uniform map (33) with an asymmetric triangle map (23) acting on each component,

U(x1,x2)=(U1(x1),U2(x2))U(x_{1},x_{2})=\left(U_{1}(x_{1}),U_{2}(x_{2})\right) (34)

where U1=tcU_{1}=t^{c} for c=0.3c=0.3 and U2=tcU_{2}=t^{c} with c=0.9c=0.9.

Figure 8 (top row) shows the two components of the map induced by the baker’s map, the checker-board distribution (bottom-left), and a histogram of 10610^{6} iterates (bottom-right) showing that the map does indeed converge in distribution to the desired distribution.

Refer to caption
Figure 8: Iterated function constructed with UU being the baker’s transformation, a histogram of iterates, and the checker board distribution. Shown is: the x1x_{1} part of the constructed map (top-left), the x2x_{2} part of the constructed map (top-right), the checker board distribution (bottom-left), and a histogram of iterates of the constructed map (bottom-right).

Figure 9 (top row) shows the two components of the map constructed using the two component-wise asymmetric triangular maps, the checker-board distribution (bottom-left), and a histogram of 10610^{6} iterates (bottom-right) showing that the map also converges in distribution to the desired distribution, and hence is also a solution to the IFPP.

Refer to caption
Figure 9: Same as Figure 8 except MM is constructed with UU being component-wise asymmetric triangular maps.

VI.3 A Numerical Construction

Numerical implementation of the factorized solution (18) is not difficult in few dimensions. In this section we present an example of numerical implementation using a normalized greyscale image of a coin 888A pre-2006 New Zealand 5050 cent coin., piecewise constant over pixels, as the target distribution; see Figure 10 (left). The marginal distributions (17) are evaluated as linear interpolation of cumulative sums over pixel values, and hence the CDF and then forward and inverse Rosenblatt transformations follow as in section IV.1. The uniform map was produced as component-wise univariate translation maps, specifically

U(x1,x2)=(U1(x1),U2(x2))U(x_{1},x_{2})=\left(U_{1}(x_{1}),U_{2}(x_{2})\right)

where U1=TcU_{1}=T^{c} for c=0.6c=0.6 and U2=TcU_{2}=T^{c} with c=0.2c=0.2. The resulting map is given by Eq. (18).

Refer to caption
Refer to caption
Figure 10: A normalized greyscale image of a coin used as the target distribution, and a normalized histogram of iterations of the map targeting this distribution. Original image (left), and normalized histogram of 10610^{6} iterations (right).

Figure 10 (right) shows a normalized histogram, binned to pixels, of 10610^{6} iterations of this map. As can be seen, the estimated PDF from the orbit of this map does reproduce the image of the coin. However, there are also obvious artifacts near the edge of the image showing that mixing could be better. We conjecture that a chaotic uniform map would produce better mixing and fewer numerical artifacts.

VII Summary and Discussion

We have shown how the solution of the IFPP, of finding an iterative map with a given invariant distribution, can be constructed from uniform maps through the factorization established in Theorem 1,

M=R1URM=R^{-1}\circ U\circ R

where RR denotes the Rosenblatt transformation that has Jacobian determinant equal to density function of the invariant distribution. In one-dimension, RR is exactly the CDF of the given distribution, so the factorization generalizes existing one-dimensional solutions to the setting of arbitrary multi-variate distributions. The factorization also shows the relationship between arbitrary iterative maps and uniform maps, i.e., given a Rosenblatt transformation the solution of the IFPP is equivalent to the choice of a uniform map that has Unif([0,1]d)\operatorname{Unif}([0,1]^{d}) as invariant distribution.

We find the factorization (18) appealing as it shows that solution of the IFPP for arbitrary distributions, and in multi dimensions, is reduced to two standard and well-studied problems, i.e., constructing the Rosenblatt transformation (or CDF in one dimension) and designing a uniform map. It is therefore surprising, to us, that the factorization (18), and more generally the Rosenblatt transformation, appears to not be widely used in the study of chaotic iterated functions and the IFPP. Grossmann and Thomae grossmann1977invariant , in one of the earliest studies of the IFPP, essentially derived the factorization (18) by introducing conjugate maps and establishing the relation (in their notation) that ρ(x)=dh1(x)/dx\rho^{*}(x)=\mathrm{d}h^{-1}(x)/\mathrm{d}x where ρ\rho^{*} is the invariant distribution and hh is the conjugating function; see (grossmann1977invariant, , Figure 3). It is a small step to identify that hh is the IDF, generalized in multi-dimensions by the inverse Rosenblatt transformation. However, the connection was not made in grossmann1977invariant , despite the Rosenblatt transformation having been already known, in statistics, for some decades rosenblatt-1952 .

We constructed solutions to the IFPP for distributions with special reflexive symmetry structure, and then with no special structure, by constructing the Rosenblatt transformation, and its inverse, for some examples in one and two dimensions. For simple distributions with analytic form the Rosenblatt transformation may be constructed analytically, while numerically-defined distributions required calculation of the marginal distributions (17) using numerical techniques.

Although this factorization and construction is applicable to high-dimensional problems, the main difficulty is obtaining all necessary marginal densities, which requires the high-dimensional integral over xk+1xdx_{k+1}\ldots x_{d} in (17). In general, this calculation can be extremely costly. Even a simple discretization of the PDF ρ\rho, or of the argument of the marginal densities (17), leads to exponential cost with dimension.

To overcome this cost, Dolgov et al. dolgov2020approximation precomputed an approximation of ρ(x1,,xd)\rho(x_{1},\ldots,x_{d}) in a compressed tensor train representation that allows fast computation of integrals in (17), and subsequent simulation of the inverse Rosenblatt transformation R1R^{-1} from the conditionals in (17), and showed that computational cost scales linearly with dimension dd. Practical examples presented in dolgov2020approximation , in dimension d32d\leq 32, demonstrate that operation by the forward and inverse Rosenblatt transformations is computationally feasible for multivariate problems with no special structure.

Finding a solution of the IFPP with desired properties is reduced to a standard problem of designing a uniform map on [0,1]d[0,1]^{d}, for which there are many existing efficient options. For example, standard computational uniform random number generators, that produce pseudo-random sequences of numbers, are one such existing uniform map, as are the quasi-Monte Carlo rules that we mentioned earlier dick2013high . These induce pseudo-random and quasi-Monte Carlo sequences, respectively, on the space XX via the inverse Rosenblatt transformation R1R^{-1} gentle2003random . Both these schemes were demonstrated in practical high-dimensional settings in dolgov2020approximation .

The RHS of Eq. (20) in d=1d=1 dimension is exactly the standard computational route for implementing inverse cumulative transformation sampling from ρ\rho, since computational uniform pseudo-random number generators perform a deterministic iteration on [0,1][0,1] to implement a uniform map gentle2003random . For d>1d>1, the RHS of Eq. (20) is the conditional distribution method that generalizes the inverse cumulative transformation method, as mentioned above devroye-rvgen-1986 ; johnson1987multivariate ; hormann-rvgen-2004 ; dolgov2020approximation . Hence Lemma 1 shows that standard computational implementation of both the inverse cumulative transformation method in d=1d=1 and the conditional distribution method in d>1d>1 is equivalent to implementing a solution to the IFFP. In this sense, computational inverse cumulative transformation sampling from ρ\rho can be viewed as the prototype for all iterative maps that target the distribution ρ\rho, with each ergodic sequence corresponding to a particular choice of uniform map.

We mentioned that the Rosenblatt transformation associated with a given distribution ρ\rho is not unique. Actually, any two Rosenblatt transformations for ρ\rho are related by a uniform map, as shown in the following Lemma.

Lemma 3.

If R1R_{1} is a Rosenblatt transformation for ρ\rho then R2R_{2} is a Rosenblatt transformation for ρ\rho if and only if

R2=UR1R_{2}=U\circ R_{1}

for some uniform map UU.

Proof: (\Rightarrow) Since R1R_{1} and R2R_{2} are Rosenblatt transformations for ρ\rho, then U=R2R11U=R_{2}\circ R_{1}^{-1} is a uniform map and R2=UR1R_{2}=U\circ R_{1}. (\Leftarrow) If R2=UR1R_{2}=U\circ R_{1} then if xρx\sim\rho, R1(x)Unif([0,1]d)R_{1}(x)\sim\operatorname{Unif}([0,1]^{d}) and UR1(x)Unif([0,1]d)U\circ R_{1}(x)\sim\operatorname{Unif}([0,1]^{d}) so R2R_{2} is a Rosenblatt transformation.  ∎

Hence, any Rosenblatt transformation RR may be written as R=UR0R=U\circ R_{0} for some uniform map UU and a fixed Rosenblatt transformation R0R_{0}.

The Rosenblatt transformations that map any distribution to the uniform distribution on the hypercube may also be used to understand mappings between spaces that are designed to transform one distribution to another, such as the ‘transport maps’ developed in parno2018transport . Consider distributions ρA\rho_{A} and ρB\rho_{B}, with Rosenblatt transformations RAR_{A} and RBR_{B}, respectively, that may be related as in the following diagram.

ρA{\rho_{A}}Unif([0,1]d){\operatorname{Unif}([0,1]^{d})}ρB{\rho_{B}}RA\scriptstyle{\displaystyle R_{A}}RA1\scriptstyle{\displaystyle R_{A}^{-1}}RB1\scriptstyle{\displaystyle R_{B}^{-1}}U\scriptstyle{\displaystyle U}RB\scriptstyle{\displaystyle R_{B}}

The diagram suggests a proof of the following lemma, that generalizes the factorization Theorem 1.

Lemma 4.

A map MM satisfies xρAM(x)ρBx\sim\rho_{A}\Rightarrow M(x)\sim\rho_{B} iff it can be written as M=RB1URAM=R_{B}^{-1}\circ U\circ R_{A}, where RAR_{A} and RBR_{B} are Rosenblatt transformations for ρA\rho_{A} and ρB\rho_{B}, respectively, and UU is a uniform map.

Hence, for given Rosenblatt transformations, the choice of a map, that maps samples from ρA\rho_{A} to samples from ρB\rho_{B}, is equivalent to the choice of a uniform map. Alternatively, if a fixed uniform map is selected, such as the identity map, the choice of map MM is completely equivalent to the choice of Rosenblatt transformations. This factorization also shows that the equivalence class of conjugate maps, noted in grossmann1977invariant , for each dimension dd, is generated by the uniform maps, and each member of the equivalence class contains maps that target each distribution, when the associated Rosenblatt transformation satisfies the mild conditions to be a conjugating function as defined in grossmann1977invariant .

References

  • [1] The density function is with respect to some underlying measure, typically Lebesgue. Throughout this paper we will use the same symbol for the distribution and associated PDF, with meaning taken from context.
  • [2] J.R. Dorfman. Cambridge Lecture Notes in Physics: An introduction to chaos in nonequilibrium statistical mechanics, volume 14. Cambridge University Press, 1999.
  • [3] Andrzej Lasota and Michael C. Mackey. Chaos, fractals, and noise. Springer-Verlag, New York, second edition, 1994.
  • [4] S. Ulam and J. von Neumann. On combination of stochastic and deterministic processes. Bulletin of the American Mathematical Society, 53(11):1120, 1947.
  • [5] R.M. May. Simple mathematical models with very complicated dynamics. Nature, 261(5560):459–467, 1976.
  • [6] J. S. Liu. Monte Carlo Strategies in Scientific Computing. Springer-Verlag, 2001.
  • [7] Stefan Klus, Péter Koltai, and Christof Schütte. On the numerical approximation of the Perron–Frobenius and Koopman operator. Journal of Computational Dynamics, 3(1):51–79, 2016.
  • [8] Siegfried Grossmann and Stefan Thomae. Invariant distributions and stationary correlation functions of one-dimensional discrete processes. Zeitschrift für Naturforschung a, 32(12):1353–1363, 1977.
  • [9] S. V. Ershov and Georgii Gennad’evich Malinetskii. The solution of the inverse problem for the Perron–Frobenius equation. USSR Computational Mathematics and Mathematical Physics, 28(5):136–141, 1988.
  • [10] F.K. Diakonos and P. Schmelcher. On the construction of one-dimensional iterative maps from the invariant density: The dynamical route to the beta distribution. Physics Letters A, 211(4):199–203, 1996.
  • [11] FK Diakonos, D. Pingel, and P. Schmelcher. A stochastic approach to the construction of one-dimensional chaotic maps with prescribed statistical properties. Physics Letters A, 264(2-3):162–170, 1999.
  • [12] D. Pingel, P. Schmelcher, and F.K. Diakonos. Theory and examples of the inverse Frobenius–Perron problem for complete chaotic maps. Chaos, 9(2):357–366, 1999.
  • [13] Erik M Bollt. Controlling chaos and the inverse Frobenius–Perron problem: global stabilization of arbitrary invariant measures. International Journal of Bifurcation and Chaos, 10(05):1033–1050, 2000.
  • [14] Xiaokai Nie and Daniel Coca. A new approach to solving the inverse Frobenius–Perron problem. In 2013 European Control Conference (ECC), pages 2916–2920. IEEE, 2013.
  • [15] Xiaokai Nie and Daniel Coca. A matrix-based approach to solving the inverse Frobenius–Perron problem using sequences of density functions of stochastically perturbed dynamical systems. Communications in Nonlinear Science and Numerical Simulation, 54:248–266, 2018.
  • [16] A. Rogers, R. Shorten, D.M. Heffernan, and D. Naughton. Synthesis of piecewise-linear chaotic maps: Invariant densities, autocorrelations, and switching. International Journal of Bifurcation and Chaos, 18(8):2169–2189, 2008.
  • [17] Nijun Wei. Solutions of the inverse Frobenius–Perron problem. Master’s thesis, Concordia University, 2015.
  • [18] Stanisław Marcin Ulam. A Collection of Mathematical Problems. Interscience Publishers, 1960.
  • [19] M. Rosenblatt. Remarks on a multivariate transformation. Ann. Math. Stat., 23(3):470–472, 1952.
  • [20] Pierre Gaspard. Chaos, scattering and statistical mechanics. Cambridge University Press, 1998.
  • [21] We have used the language of differential maps, as all the maps that we display in this paper are differentiable almost everywhere [varberg1971change]. More generally, |J(x)|1|J(x)|^{-1} denotes the density of ρnM1\rho_{n}M^{-1} with respect to ρn+1\rho_{n+1}, see, e.g., [3, Remark 3.2.4.].
  • [22] Hence TcT^{c} is the translation operator on the unit circle S1S^{1}.
  • [23] G Györgyi and P Szépfalusy. Fully developed chaotic 1-d maps. Zeitschrift für Physik B Condensed Matter, 55(2):179–186, 1984.
  • [24] M.E. Johnson. Multivariate Statistical Simulation. John Wiley & Sons, 1987.
  • [25] L. Devroye. Non-Uniform Random Variate Generation. Springer-Verlag, 1986.
  • [26] W. Hörmann, J. Leydold, and G. Derflinger. Automatic Nonuniform Random Variate Generation. Springer-Verlag, 2004.
  • [27] Sergey Dolgov, Karim Anaya-Izquierdo, Colin Fox, and Robert Scheichl. Approximation and sampling of multivariate probability distributions in the tensor train decomposition. Statistics and Computing, 30(3):603–625, 2020.
  • [28] When UU satisfies the stronger condition that Unif([0,1]d)\operatorname{Unif}([0,1]^{d}) is the equilibrium distribution, UU is called an exact map [3].
  • [29] Josef Dick, Frances Y Kuo, and Ian H Sloan. High-dimensional integration: the quasi-Monte Carlo way. Acta Numerica, 22:133, 2013.
  • [30] The two period sawtooth map s2s_{2} is also called the Bernoulli map, and its orbit 𝒪+(x)\mathcal{O}^{+}(x) is the dyadic transformation.
  • [31] This is the ‘broken linear transformation’ in [8] of order p=2p=2\ell.
  • [32] Alan Rogers, Robert Shorten, and Daniel M Heffernan. Synthesizing chaotic maps with prescribed invariant densities. Physics Letters A, 330(6):435–441, 2004.
  • [33] W. Huang. Characterizing chaotic processes that generate uniform invariant density. Chaos, Solitons & Fractals, 25(2):449–460, 2005.
  • [34] Computation was performed in MatLab that implements IEEE Standard 754 for double-precision binary floating-point format.
  • [35] A pre-2006 New Zealand 5050 cent coin.
  • [36] James E Gentle. Random number generation and Monte Carlo methods, volume 381. Springer, 2003.
  • [37] Matthew D Parno and Youssef M Marzouk. Transport map accelerated Markov chain Monte Carlo. SIAM/ASA Journal on Uncertainty Quantification, 6(2):645–682, 2018.