Invariant ideals and its applications to the turnpike theory

The turnpike theory investigates an important property of dynamical systems. It can be considered as a theory that justifies the importance of some equilibrium/stationary states. For example, in macroeconomic models the turnpike property states that, regardless of initial conditions, all optimal trajectories spend most of the time within a small neighborhood of some optimal stationary point when the planning period is long enough. Obviously, in the absence of such a property, using some of optimal stationary points as a criteria for “good” policy formulation might be misleading. Correspondingly, the turnpike property is in the core of many important theories in economics.

Many real-life processes are happening in an optimal way and have the tendency to stabilize; that is, the turnpike property is expected to hold for a broad class of problems. It provides valuable insights into the nature of these processes by investigating underlying principles of evolution that lead to stability. It can also be used to assess the “quality” of mathematical modeling and to develop more adequate equations describing system dynamics as well as optimality criteria.

The first result in this area is obtained by John von Neumann ([30]) for discrete time systems. The phenomenon is called the turnpike property after Chapter 12, [9] by Dorfman, Samuelson and Solow. For a classification of different definitions for this property, see [2, 22, 28, 36], as well as [6] for the so-called exponential turnpike property. Possible applications in Markov Games can be found in a recent study [16].

The approaches suggested for the study of the turnpike property involve continuous and discrete time systems. Some convexity assumptions are sufficient for discrete time systems [22, 28]; however, rather restrictive assumptions are usually required for continuous time systems. The majority of them deal with the (discounted and undiscounted) integral functionals. We mention here the approaches developed by Rockafellar [33, 34], Scheinkman, Brock and collaborators (see, for example, [21, 35]), Cass and Shell [4], Leizarowitz [18], Mamedov [24], Montrucchio [29], Zaslavski [37, 38, 39] (we refer to [2, 36] for more references).

In this paper we consider an optimal control problem in discrete time. It extends the results obtained in [23] where a special class of terminal functionals is introduced as a lower limit at infinity of utility functions. This approach allowed to establish the turnpike property for a much broader class of optimal control problems than those involving integral functionals (discounted and undiscounted).

Later, this class of terminal functionals was used to establish a connection between the turnpike theory and the notion of statistical convergence [25, 32]; as a result, the convergence of optimal trajectories is proved in terms of the statistical (“almost”) convergence. These terminal functionals also allowed the extension of the turnpike theory to time delay systems; the first results in this area have been established in several recent papers [13, 26]. Moreover, some generalizations based on the notion of the $A$-statistical cluster points have been obtained in [7].

The main purpose of this paper is to formulate the optimality criteria by using the notion of ideal convergence. As detailed in the next section, the ideal convergence is a more general concept than the statistical convergence as well as the $A$-statistical convergence. In this way the turnpike property is established for a broad class of non-convex optimal control problems where the asymptotical stability of optimal trajectories is formulated in terms of the ideal convergence.

Recently (and independently) Leonetti and Caprio in [19] considered turnpike property for ideals invariant under translation in the context of normed vector spaces. We discuss our approaches in Section 4.

The rest of this article is organized as follows. In the next section the definition of the ideal, its properties and some particular cases, including the statistical convergence, are provided. In Section 3 we formulate the optimal control problem and main assumptions. The main results of the paper — the turnpike theorems are provided in Section 4. The proof of the main theorem is in Section 5.

Let $x=(x_{n})_{n\in\mathbb{N}}$ be a sequence of elements of $\mathbb{R}^{m}$. For the sake of simplicity, we will consider the Euclidean norm $\left\|\cdot\right\|.$ The classical definition of convergence of $x$ to $a$ says that for every $\varepsilon>0$ the set of all $n\in\mathbb{N}$ with $\left\|x_{n}-a\right\|\geq\varepsilon$ is finite, i.e. it is “small” in some sense. If we understand the word “small” as “of asymptotic density zero” then we obtain the definition of statistical convergence (Def. 6). The same method can be used to formulate the definition of statistical cluster point. The classical one says that $a$ is a cluster point of $x$ if for every $\varepsilon>0$ the set of all $n\in\mathbb{N}$ with $\left\|x_{n}-a\right\|<\varepsilon$ is infinite, i.e. it has “many” elements. If “many” means “not of asymptotic density zero” then we obtain the definition of statistical cluster point (see e.g. [12]).

One of the possible generalizations of this kind of being “small” (having “many” elements) is “belonging to the ideal” (“be an element of co-ideal”).

The cardinality of a set $X$ is denoted by ${\#{}X}$. $\mathcal{P}(\mathbb{N})$ denotes the power set of $\mathbb{N}$.

Ideals $\mathcal{I}_{d}$ and $\mathcal{I}_{log}$ belongs to the wider class of Erdős-Ulam ideal’s (defined by submeasures of special kind, see [14]). Ideal $\mathcal{I}_{1/n}$ is an representant of the class of summable ideals (see [27]). The fact that $\mathcal{W}$ is an ideal follows from the non-trivial theorem of van der Waerden (this ideal was considered by Kojman in [15]). One can also consider trivial ideals $\mathcal{I}=\mathcal{P}(\mathbb{N})$, $\mathcal{I}=\{\emptyset\}$, or principal ideals $\mathcal{I}_{n}=\{A\subset\mathbb{N}\colon n\notin A\}$, however they are not interesting from our point of view. If not explicitly said we assume that all considered ideals are proper (i.e. $\mathcal{I}\not=\mathcal{P}(\mathbb{N})$) and contain all finite sets (i.e. $\textrm{Fin}\subset\mathcal{I}$). The inclusions between abovementioned families are shown on Figure 1. The only non-trivial inclusions are: $\mathcal{I}_{1/n}\subset\mathcal{I}_{d}$ (a folklore application of Cauchy condensation test), $\mathcal{W}\subset\mathcal{I}_{d}$ (the famous theorem of Szemerédi), and $\mathcal{I}_{d}\subset\mathcal{I}_{log}$ (by well-known inequalities between upper logarithmic density and upper asymptotic density). It is easy to observe that $\mathcal{I}_{1/n}\not\subset\mathcal{W}$, but the status of the inclusion $\mathcal{W}\subset\mathcal{I}_{1/n}$ is unknown (“Erdős conjecture on arithmetic progressions” says that the van der Waerden ideal $\mathcal{W}$ is contained in the ideal $\mathcal{I}_{1/n}$.)

Consider the problem

Here $\zeta^{0}$ is a fixed initial point, function $f\colon\mathbb{R}^{m}\times\mathbb{R}^{t}\to\mathbb{R}^{m}$ is continuous, $U\subset\mathbb{R}^{t}$ is a compact set, $\phi\colon\mathbb{R}^{m}\to\mathbb{R}$ is a continuous function, and for any sequence of reals $y=(y_{n})$

The pair $\langle u,x\rangle$ is called a process if the sequences $x=(x_{n})$ and $u=(u_{n})$ satisfy $(*)$ for all $n\in\mathbb{N}$ ($x$ is called a trajectory and $u$ is called a control).

In the sequel we will use the following characterization of the functional $J_{\mathcal{I}}$ ([7, Le. 4.1]) that is a generalization of Lemma 3.1 in [32] established for the statistical convergence, as well as the corresponding result from [20] established for classical convergence (see also [19, Cor. 3.3]).

We assume that there is a compact (bounded and closed) set $C\subset\mathbb{R}^{m}$ such that $x_{n}\in C$ for all trajectories; that is, we assume that trajectories are uniformly bounded.

$\zeta\in\mathbb{R}^{m}$ is called a stationary point if there exists $u_{0}\in U$ such that $f(\zeta,u_{0})=\zeta$. We denote the set of stationary points by $M$. It is clear that $M$ is a closed set. $\zeta^{\star}\in M$ is called an optimal stationary point if

We will assume that the set of all optimal stationary points is non-empty. This is not a restrictive assumption since function $\phi$ is continuous and the set $M$ is closed; for example, it is satisfied if $M$ is in addition bounded.

Define the set

and

We assume that the set $C$ is large enough to accommodate $M^{\star};$ that is, $M^{\star}\subset C$. Then clearly, $M^{\star}=M\cap D^{\star}$.

Consider the following three conditions.

• $(C1)$:

  optimal stationary point $\zeta^{\star}$ is unique, i.e. $M^{\star}=\{\zeta^{\star}\}$;

• $(\mathcal{I}{\rm/}C2)$:

  there exists a process $\langle u^{\star},x^{\star}\rangle$ such that $\Gamma_{\mathcal{I}}(x^{\star})\subset D^{\star}$;

• $(C3)$:

  there exists a continuous function $P\colon\mathbb{R}^{m}\to\mathbb{R}$ such that

  $$P(f(x_{0},u_{0}))<P(x_{0})~{}~{}~{}~{}~{}{\rm for~{}all}~{}~{}x_{0}\in D^{\star}\setminus M^{\star},u_{0}\in U,$$

  and

  $$P(f(x_{0},u_{0}))\leq P(x_{0})~{}~{}~{}~{}~{}{\rm for~{}all}~{}~{}x_{0}\in D^{\star},u_{0}\in U.$$

One can also consider condition $(C2)=(\textrm{Fin}{\rm/}C2)$:

• $(C2)$:

  there exists a process $\langle u^{\star},x^{\star}\rangle$ such that any limit point of the sequence $x^{\star}$ is in $D^{\star}$.

Note that if the (unique) optimal stationary point $\zeta^{\star}$ belongs to the interior of $D^{\star}$, then proof of turnpike property is not difficult and can be regarded as a “trivial” case where condition $(C3)$ ensures the existence of some Lyapunov function, with derivative $P,$ defined on a small neighborhood of $\zeta^{\star}.$

The most interesting case is when an optimal stationary point $\zeta^{\star}$ belongs to a boundary of $D^{\star}$; that is, the both sets $D^{\star}$ and $D^{\star-}=\left\{\zeta\in C\colon~{}\phi(\zeta)<\phi^{\star}\right\}$ have nonempty intersection with any small neighborhood of $\zeta^{\star}$. In this case the inequality $P(f(x_{0},u_{0}))>P(x_{0})$ may hold for some $x_{0}\in D^{\star-}$; that is, condition $(C3)$ does not guarantee the existence of Lyapunov functions.

Note also that condition $(\mathcal{I}{\rm/}C2)/(C2)$ can be formulated equivalently “there exists a process $\langle u^{\star},x^{\star}\rangle$ such that $J_{\mathcal{I}}(x^{\star})\geq\phi^{\star}$” (see [19, $(A6)$]), or stronger “there exists a process $\langle u^{\star},x^{\star}\rangle$ such that $x^{\star}\to_{\mathcal{I}}\zeta^{\star}$” (see [23, 25, 32]),

Recall that if $\mathcal{I}\subset\mathcal{J}$ then $\mathcal{I}$-convergence is stronger than $\mathcal{J}$-convergence, thus by Proposition 5, $(C2)$ is stronger than $(\mathcal{I}{\rm/}C2)$ for each non-trivial $\mathcal{I}$ which is invariant under translations. Example 13 shows that these two conditions are really different, i.e. there exists a system for which $(C1)$, $(\mathcal{I}_{d}{\rm/}C2)$, $(C3)$ hold, but $(C2)$ does not hold (see also [19, Ex. 2.5]).

The main result of this paper is presented next. The proof of this theorem is provided in Section 5.

Note that from part $(3)$ of Proposition 5 the assertion “$x^{opt}\to_{\mathcal{I}}\zeta^{\star}$” is equivalent to “$\Gamma_{\mathcal{I}}(x)=\{\zeta^{\star}\}$”.

It is also easy to see that the assertion of Theorem 14 is true if $D^{\star}$ is a singleton (i.e. $D^{\star}=\{\zeta^{\star}\}$). However, the following example shows that if $\zeta^{\star}$ is an isolated point of $D^{\star}$ (if we assume only the first part of condition $(C3)$) then Theorem 14 may not be true.

Example 15 works for classical convergence, statistical convergence, and for general ideal convergence. It shows that additional assumption about “density” of $D^{\star}$ in $\zeta^{\star}$ (i.e. the second part of condition $(C3)$) is necessary in [7], as well as in [25].

Recently Leonetti and Caprio in [19] proposed another way to bypass the problem indicated in the Example 15:

• $(C3{\textrm{-LC}})$:

  there exists a linear (and therefore continuous) function $P\colon\mathbb{R}^{m}\to\mathbb{R}$ such that

  $$P(f(x_{0},u_{0}))<P(x_{0})~{}~{}~{}~{}~{}{\rm for~{}all}~{}~{}x_{0}\in D^{\star},u_{0}\in U,\langle x_{0},f(x_{0},u_{0})\rangle\not=\langle\zeta^{\star},\zeta^{\star}\rangle,$$

where $\zeta^{\star}$ is an optimal stationary point. It follows from the above condition that $\zeta^{\star}$ is the unique optimal stationary point, and it is easy to see that $(C3{\textrm{-LC}})$ imply $(C3)$. However, we do not have any example of the system with $(C1)+(C3)$ and without $(C3{\textrm{-LC}})$.

For every $r\in\mathbb{R}$ define the set

Clearly, $D^{\star}=D_{\phi^{\star}}$. For any continuous function $P\colon\mathbb{R}^{m}\to\mathbb{R}$ let

and

It is clear that $M\cap E_{P}=\emptyset$. If $A\subset\mathbb{R}^{m}$ is compact then

Analogously we define operator $\arg\max$.

Proof.  As $J_{\mathcal{I}}(x)\geq r$, by Lemma 12, $J_{\mathcal{I}}(x)=\min_{\zeta\in\Gamma_{\mathcal{I}}(x)}\phi(\zeta)\geq r$. Thus $\Gamma_{\mathcal{I}}(x)\subset D_{r}$, and so $\arg\min_{\zeta\in\Gamma_{\mathcal{I}}(x)}P(\zeta)\subset D_{r}$.

Let

It is clear that $F\colon\mathbb{R}^{m}\to\mathbb{R}$ is continuous, and

Suppose that there exists $\zeta_{1}\in\Gamma_{\mathcal{I}}(x)$ such that $\zeta_{1}\in E_{P}$ and

Denote $\delta=-F(\zeta_{1})/8.$ Clearly $\delta>0$ thanks to (2).

Since functions $F$ and $P$ are continuous and $\Gamma_{\mathcal{I}}(x)$ is a compact set, there exists $\gamma>0$ such that

If $x_{k}\in B(\zeta_{1},\gamma)$ then $F(x_{k})\leq-4\delta$, i.e. $P(f(x_{k},u_{0}))\leq P(x_{k})-4\delta$ for each $u_{0}\in U$ and in particular for $u_{k}\in U$ that leads to $P(x_{k+1})\leq P(x_{k})-4\delta.$ Moreover, from (4) we have $P(x_{k})\leq P(\zeta_{1})+\delta$ and therefore