Hitting Times for Continuous-Time Imprecise-Markov Chains

We now turn to stochastic processes, which are fundamentally the subject of this work. The typical (measure-theoretic) way to define a stochastic process is simply as a family $(X_{i})_{i\in\mathcal{I}}$ of random variables with index set $\mathcal{I}$. This index set represents the time domain of the stochastic process. The random variables are understood to be taken with respect to some underlying probability space $(\Omega_{\mathcal{I}},\mathcal{F}_{\mathcal{I}},P)$, where $\Omega_{\mathcal{I}}$ is a set of sample paths, which are functions from $\mathcal{I}$ to $\mathcal{X}$ representing possible realizations of the evolution of the underlying process through $\mathcal{X}$. The random variables $X_{i}$, $i\in\mathcal{I}$ are canonically the maps $X_{i}:\omega\mapsto\omega(i)$ on $\Omega_{\mathcal{I}}$.

However, for our purposes it will be more convenient to instead refer to the probability measure $P$ as the stochastic process. Different processes $P$ may then be taken over the same measurable space $(\Omega_{\mathcal{I}},\mathcal{F}_{\mathcal{I}})$, using the same canonical variables $(X_{i})_{i\in\mathcal{I}}$ for all these processes.

In this work we will use both discrete- and continuous-time stochastic processes, which corresponds to choosing $\mathcal{I}=\mathbb{N}_{0}$ or $\mathcal{I}=\mathbb{R}_{\geq 0}$, respectively. In both cases we take $\mathcal{F}_{\mathcal{I}}$ to be the $\sigma$-algebra generated by the cylinder sets; this ensures that all functions that we consider are measurable.

In the discrete-time case, we let $\Omega_{\mathbb{N}_{0}}$ be the set of all functions from $\mathbb{N}_{0}$ to $\mathcal{X}$. A discrete-time stochastic process $P$ is then simply a probability measure on $(\Omega_{\mathbb{N}_{0}},\mathcal{F}_{\mathbf{\mathbb{N}_{0}}})$. Moreover, $P$ is said to be a Markov chain if it satisfies the (discrete-time) Markov property, meaning that

for all $x_{0},\ldots,x_{n+1}\in\mathcal{X}$ and $n\in\mathbb{N}_{0}$. If, additionally, it holds for all $x,y\in\mathcal{X}$ and $n\in\mathbb{N}_{0}$ that

then $P$ is said to be a (time-)homogeneous Markov chain. We use $\mathbb{P}_{\mathbb{N}_{0}},\mathbb{P}_{\mathbb{N}_{0}}^{\mathrm{M}}$, and $\mathbb{P}_{\mathbb{N}_{0}}^{\mathrm{HM}}$ to denote, respectively, the set of all discrete-time stochastic processes; the set of all discrete-time Markov chains; and the set of all discrete-time homogeneous Markov chains.

In the continuous-time case, we let $\Omega_{\mathbb{R}_{\geq 0}}$ be the set of all cadlag functions from $\mathbb{R}_{\geq 0}$ to $\mathcal{X}$. A continuous-time stochastic process $P$ is a probability measure on $(\Omega_{\mathbb{R}_{\geq 0}},\mathcal{F}_{\mathbb{R}_{\geq 0}})$. The process $P$ is said to be a Markov chain if it satisfies the (continuous-time) Markov property,

for all $x_{t_{0}},\ldots,x_{t_{n+1}}\in\mathcal{X}$, $t_{0}<\cdots<t_{n}\leq t_{n+1}\in\mathbb{R}_{\geq 0}$, and all $n\in\mathbb{N}_{0}$. If, additionally, it holds that

for all $x,y\in\mathcal{X}$ and all $t,s\in\mathbb{R}_{\geq 0}$ with $t\leq s$, then $P$ is said to be a (time-)homogeneous Markov chain. We use $\mathbb{P}_{\mathbb{R}_{\geq 0}},\mathbb{P}_{\mathbb{R}_{\geq 0}}^{\mathrm{M}}$, and $\mathbb{P}_{\mathbb{R}_{\geq 0}}^{\mathrm{HM}}$ to denote, respectively, the set of all continuous-time stochastic processes; the set of all continuous-time Markov chains; and the set of all continuous-time homogeneous Markov chains.

We refer to [Norris, 1998] for an excellent further introduction to discrete-time and continuous-time Markov chains.

Throughout this work, we make extensive use of operator-theoretic representations of the behavior of stochastic processes, and Markov chains in particular. The first reason for this is that such operators serve as a way to parameterize Markov chains. Moreover, they are also useful as a computational tool, since they can often be used to express inferences of interest; see, e.g., Propositions 1 and 2 further on. We introduce the basic concepts below, and refer to e.g. [Norris, 1998] for details.

A transition matrix $T$ is a linear operator on $\smash{\mathbb{R}^{\mathcal{X}}}$ such that, for all $x\in\mathcal{X}$, it holds that $T(x,y)\geq 0$ for all $y\in\mathcal{X}$, and $\sum_{y\in\mathcal{X}}T(x,y)=1$. There is an important and well-known connection between Markov chains and transition matrices; for any discrete-time homogeneous Markov chain $P$, we can define the corresponding transition matrix ${}^{P}T$ as

Since $P$ is a probability measure, we clearly have that ${}^{P}T$ is a transition matrix. Conversely, a given transition matrix $T$ uniquely determines a discrete-time homogeneous Markov chain $P$ with ${}^{P}T=T$, up to the specification of the initial distribution $P(X_{0})$. For this reason, transition matrices are often taken as a crucial parameter to specify (discrete-time, homogeneous) Markov chains.

Analogously, for a (non-homogeneous) discrete-time Markov chain $P$, we might define a family $({}^{P}T_{n})_{n\in\mathbb{N}_{0}}$ of time-dependent corresponding transition matrices, with

for all $x,y\in\mathcal{X}$ and $n\in\mathbb{N}_{0}$. Conversely, any family $(T_{n})_{n\in\mathbb{N}_{0}}$ of transition matrices uniquely determines a discrete-time Markov chain $P$ with ${}^{P}T_{n}=T_{n}$ for all $n\in\mathbb{N}_{0}$, again up to the specification of $P(X_{0})$.

In the continuous-time setting, transition matrices are also of great importance. However, it will be instructive to first introduce rate matrices. A rate matrix $Q$ is a linear operator on $\smash{\mathbb{R}^{\mathcal{X}}}$ such that, for all $x\in\mathcal{X}$, it holds that $Q(x,y)\geq 0$ for all $y\in\mathcal{X}$ with $x\neq y$, and $\sum_{y\in\mathcal{X}}Q(x,y)=0$.

For any rate matrix $Q$ and any $t\in\mathbb{R}_{\geq 0}$, the matrix exponential $e^{Qt}$ of $Qt$ can be defined as [Van Loan, 2006]

An alternative characterization is as the (unique) solution to the matrix ordinary differential equation [Van Loan, 2006]

For any $t,s\in\mathbb{R}_{\geq 0}$ it holds that $e^{Q(t+s)}=e^{Qt}e^{Qs}$, and we immediately have $e^{Q0}=I$. The family $(e^{Qt})_{t\in\mathbb{R}_{\geq 0}}$ is therefore called the semigroup generated by $Q$, and $Q$ is called the generator of this semigroup. Moreover, for any rate matrix $Q$ and any $t\in\mathbb{R}_{\geq 0}$, $e^{Qt}$ is a transition matrix [Norris, 1998, Thm 2.1.2].

Now let us consider a continuous-time homogeneous Markov chain $P$, and define the corresponding transition matrix¹¹1Note that in continuous-time, we always have to measure the transition-time interval $[0,t]$ to specify these matrices. ${}^{P}T_{t}$ for all $t\in\mathbb{R}_{\geq 0}$ and $x,y\in\mathcal{X}$ as

It turns out that there is then a unique rate matrix ${}^{P}\!Q$ associated with $P$ such that ${}^{P}T_{t}=e^{{}^{P}\!Qt}$ for all $t\in\mathbb{R}_{\geq 0}$. By combining Equations (1) and (2), we can identify ${}^{P}\!Q$ as

As before, in the other direction we have that any fixed rate matrix $Q$ uniquely determines a continuous-time homogeneous Markov chain $P$ with ${}^{P}\!Q=Q$, up to the specification of $P(X_{0})$. For this reason, rate matrices are often used to specify (continuous-time, homogeneous) Markov chains.

Let us finally consider a (not-necessarily homogeneous) continuous-time Markov chain $P$. For any $t,s\in\mathbb{R}_{\geq 0}$ with $t\leq s$, we can then define a transition matrix ${}^{P}T_{t}^{s}$ with, for all $x,y\in\mathcal{X}$, ${}^{P}T_{t}^{s}(x,y)\coloneqq P(X_{s}=y\,|\,X_{t}=x)$. Under appropriate assumptions of differentiability, this induces a family $({}^{P}\!Q_{t})_{t\in\mathbb{R}_{\geq 0}}$ of rate matrices ${}^{P}\!Q_{t}$, as

In the converse direction we might try to reconstruct the transition matrices of $P$ by solving the matrix ordinary differential equation(s)

By comparing with Equation (1), we see that in the special case where ${}^{P}\!Q_{s}$ does not depend on $s$—that is, where $P$ is homogeneous with ${}^{P}\!Q_{s}={}^{P}\!Q$, say—we indeed obtain ${}^{P}T_{t}^{s}=e^{{}^{P}\!Q(s-t)}$. However, in general the non-autonomous system (4) does not have such a closed-form solution, and we cannot move beyond this implicit characterization.

We now have all the pieces to introduce the inference problem that is the subject of this work, viz. the expected hitting times of some non-empty set of states $A\subset\mathcal{X}$ with respect to a particular stochastic process. We take this set $A$ to be fixed for the remainder of this work.

In the discrete-time case, we consider the (extended real-valued)²²2We agree that $0(+\infty)=0$; $(+\infty)+(+\infty)=+\infty$; and, for any $c\in\mathbb{R}$, $(+\infty)+c=+\infty$ and $c(+\infty)=+\infty$ if $c>0$. function $\tau_{\mathbb{N}_{0}}:\Omega_{\mathbb{N}_{0}}\to\mathbb{R}_{\geq 0}\cup\{+\infty\}$ given by

This captures the number of steps before a process $P$ “hits” any state in $A$. The expected hitting time for a discrete-time process $P$ starting in $x\in\mathcal{X}$ is then defined as

We use $\mathbb{E}_{P}\bigl{[}\tau_{\mathbb{N}_{0}}\,|\,X_{0}\bigr{]}$ to denote the extended real-valued function on $\mathcal{X}$ given by $x\mapsto\mathbb{E}_{P}\bigl{[}\tau_{\mathbb{N}_{0}}\,|\,X_{0}=x\bigr{]}$. When dealing with homogeneous Markov chains, this quantity has the following simple characterization:

In the continuous-time case, the definition is analogous; we introduce a function $\tau_{\mathbb{R}_{\geq 0}}:\Omega_{\mathbb{R}_{\geq 0}}\to\mathbb{R}_{\geq 0}\cup\{+\infty\}$ as

This function measures the time until a process “hits” any state in $A$ on a given sample path. The expected hitting time for a continuous-time process $P$ starting in $x\in\mathcal{X}$ is

We again use $\smash{\mathbb{E}_{P}\bigl{[}\tau_{\mathbb{R}_{\geq 0}}\,|\,X_{0}\bigr{]}}$ to denote the extended-real valued function on $\mathcal{X}$ given by $x\mapsto\smash{\mathbb{E}_{P}\bigl{[}\tau_{\mathbb{R}_{\geq 0}}\,|\,X_{0}=x\bigr{]}}$. Also in this case, the characterization for homogeneous Markov chains is particularly simple:

We already mentioned that an imprecise-Markov chain is essentially simply a set $\mathcal{P}$ of stochastic processes. Let us now consider how to define such sets.

We start by considering the discrete-time case; then, clearly, $\mathcal{P}$ will be a set of discrete-time processes. We will parameterize such a set with some non-empty set $\mathcal{T}$ of transition matrices. Our aim is then to include in $\mathcal{P}$ all processes that are in some sense “compatible” with $\mathcal{T}$.⁵⁵5We will not constrain the initial models $P(X_{0})$ of the elements of $\mathcal{P}$, since in any case such a choice would not influence the inferences that we study in this work. However, at this point we are faced with a choice about which type of processes to include in this set, and these different choices lead to different types of imprecise-Markov chains.

Arguably the conceptually most simple model is $\mathcal{P}^{\mathrm{HM}}_{\mathcal{T}}$, which contains all homogeneous Markov chains $P$ whose corresponding transition matrix is included in $\mathcal{T}$:

However, we could instead consider $\mathcal{P}^{\mathrm{M}}_{\mathcal{T}}$, which is the set of all (not-necessarily homogeneous) Markov chains whose time-dependent transition matrices are contained in $\mathcal{T}$:

The last choice that we consider here is the set $\mathcal{P}^{\mathrm{I}}_{\mathcal{T}}$, which essentially contains all discrete-time processes whose single-step transition dynamics are described by $\mathcal{T}$. Its characterization is more cumbersome since we have not expressed these general processes in terms of transition matrices, but we can say that it is the set of all $P\in\mathbb{P}_{\mathbb{N}_{0}}$ such that for all $n\in\mathbb{N}_{0}$ and all $x_{0},\ldots,x_{n}\in\mathcal{X}$, there is some $T\in\mathcal{T}$ such that for all $y\in\mathcal{X}$ it holds that

This last type is called an imprecise-Markov chain under epistemic irrelevance, whence the superscript ‘$\mathrm{I}$’.

Note that the three types $\mathcal{P}^{\mathrm{HM}}_{\mathcal{T}},\mathcal{P}^{\mathrm{M}}_{\mathcal{T}}$, and $\mathcal{P}^{\mathrm{I}}_{\mathcal{T}}$ capture not only “plausible” variation in terms of parameter uncertainty—expressed through the set $\mathcal{T}$—but also variation in terms of the structural independence conditions that we consider! So, from an applied perspective, if someone is not sure whether the underlying system that they are studying is truly Markovian and/or time-homogeneous, they might choose to use different such sets in their analysis.

In the continuous-time case, we again proceed analogously. First, we fix a non-empty set $\mathcal{Q}$ of rate matrices, which will be the parameter for our models. We then first consider the set $\mathcal{P}^{\mathrm{HM}}_{\mathcal{Q}}$ of all homogeneous Markov chains whose rate matrix is included in $\mathcal{Q}$:

The other two types are constructed in analogy to the discrete-time case, but unfortunately we don’t have the space for a complete exposition of their characterization. Instead we refer the interested reader to [Krak et al., 2017, Krak, 2021] for an in-depth study of these different types and comparisons between them; in what follows we limit ourselves to a largely intuitive specification.

The model $\mathcal{P}^{\mathrm{M}}_{\mathcal{Q}}$ is the set of all continuous-time (not-necessarily homogeneous) Markov chains whose transition dynamics are compatible with $\mathcal{Q}$ at every point in time. This includes in particular all Markov chains $P$ satisfying the appropriate differentiability assumptions to meaningfully say that the time-dependent rate matrices ${}^{P}\!Q_{t}$—as in Equation (3)—are included in $\mathcal{Q}$ for all $t\in\mathbb{R}_{\geq 0}$. However, $\mathcal{P}^{\mathrm{M}}_{\mathcal{Q}}$ also contains other processes that are not (everywhere) differentiable; see e.g. [Krak, 2021, Sec 4.6 and 5.2] for the technical details.

The most involved model to explain is again $\mathcal{P}^{\mathrm{I}}_{\mathcal{Q}}$, which includes all continuous-time processes whose time- and history-dependent transition dynamics can be described using elements of $\mathcal{Q}$. It includes, but is not limited to, appropriately differentiable processes $P$ such that for all $n\in\mathbb{N}_{0}$, all $t_{0}<\cdots<t_{n}\in\mathbb{R}_{\geq 0}$, and all $x_{t_{0}},\ldots,x_{t_{n}}\in\mathcal{X}$, there is some $Q\in\mathcal{Q}$ such that for all $y\in\mathcal{X}$ it holds that

We again refer to [Krak, 2021, Sec 4.6 and 5.2] for the technical details involving the additional elements of $\mathcal{P}^{\mathrm{I}}_{\mathcal{Q}}$ that are not appropriately differentiable. Importantly, we note the nested structure  [Krak, 2021, Prop 5.9]

where the inclusions are strict provided $\mathcal{Q}$ isn’t trivial.

For notational convenience, we will use identical sub- and superscripts to denote the corresponding lower- and upper expectations for any of these imprecise-Markov chains; e.g., we let $\underline{\mathbb{E}}^{\mathrm{HM}}_{\mathcal{T}}[\cdot\,|\,\cdot]\coloneqq\underline{\mathbb{E}}_{\mathcal{P}^{\mathrm{HM}}_{\mathcal{T}}}[\cdot\,|\,\cdot]$.

Let us now introduce some machinery to describe the dynamics of imprecise-Markov chains. In particular, we here move from the set-valued parameters $\mathcal{T}$ and $\mathcal{Q}$ used in Section 3.1, to their dual representations; these are operators that can serve as computational tools.

In Section 3.1, we described discrete-time imprecise-Markov chains using non-empty sets $\mathcal{T}$ of transition matrices. With any such set, we can associate the corresponding lower- and upper transition operators $\underline{T}$ and $\overline{T}$ on $\smash{\mathbb{R}^{\mathcal{X}}}$, defined respectively as

More generally, any operator $\underline{T}$ (resp. $\overline{T}$) on $\smash{\mathbb{R}^{\mathcal{X}}}$ is a lower (resp. upper) transition operator if for all $f,g\in\smash{\mathbb{R}^{\mathcal{X}}}$, all $\lambda\in\mathbb{R}_{\geq 0}$, and all $x\in\mathcal{X}$, it holds that [De Bock, 2017]

1. 1.

   $\min_{y\in\mathcal{X}}f(y)\leq\underline{T}f(x)$ and $\overline{T}f(x)\leq\max_{y\in\mathcal{X}}f(y)$

2. 2.

   $\underline{T}f+\underline{T}g\leq\underline{T}(f+g)$ and $\overline{T}(f+g)\leq\overline{T}f+\overline{T}g$

3. 3.

   $\underline{T}(\lambda f)=\lambda\underline{T}f$ and $\overline{T}(\lambda f)=\lambda\overline{T}f$.

It should be noted that lower- and upper transition operators are conjugate, in that any $\underline{T}$ induces a corresponding upper transition operator $\overline{T}(\cdot)=-\underline{T}(-\cdot)$, and vice versa. Moreover, any transition matrix $T$ is also a lower—and, by its linearity, upper—transition operator.

It is easily verified that the lower- and upper transition operators corresponding to a given non-empty set $\mathcal{T}$ are, indeed, lower- and upper transition operators. Conversely, with a given lower transition operator $\underline{T}$, we can associate the set of transition matrices that dominate it, in the sense that

This set satisfies the following important properties:

Notably, there is a one-to-one relation between non-empty sets of transition matrices that are closed and convex and have separately specified rows, and lower (or upper) transition operators: if $\underline{T}$ is the lower transition operator for the set $\mathcal{T}$, and if $\mathcal{T}$ satisfies these properties, then $\mathcal{T}=\mathcal{T}_{\underline{T}}$ [Krak, 2021, Cor 3.38]. Hence these objects may serve as dual representations for each other.

One reason that this is important is the use of $\underline{T}$ as a computational tool; under the conditions of this duality it holds that for any function $f\in\smash{\mathbb{R}^{\mathcal{X}}}$ and any $n\in\mathbb{N}_{0}$, we can write [Hermans and Škulj, 2014]

where $\underline{T}$ is the lower transition operator for $\mathcal{T}$. This reduces the problem of computing such inferences for the imprecise-Markov chains $\mathcal{P}^{\mathrm{M}}_{\mathcal{T}}$ and $\mathcal{P}^{\mathrm{I}}_{\mathcal{T}}$ to solving $n$ independent linear optimization problems over $\mathcal{T}$; first compute $f_{1}\coloneqq\underline{T}f$, then compute $f_{2}\coloneqq\underline{T}\,f_{1}=\underline{T}^{2}f$, and so forth. Note that this method in general only yields a conservative bound on the corresponding inference for $\mathcal{P}^{\mathrm{HM}}_{\mathcal{T}}$, as the minimizers $T_{k}$ that obtain $T_{k}f_{k-1}=\underline{T}f_{k-1}$ may be different at each step.

We next consider the dynamics in the continuous-time setting. We proceed analogously to the above: we first consider a non-empty and bounded⁷⁷7In the induced operator norm. set $\mathcal{Q}$ of rate matrices. With this set, we then associate the corresponding lower- and upper rate operators $\underline{Q}$ and $\overline{Q}$ on $\smash{\mathbb{R}^{\mathcal{X}}}$, defined as

More generally, any operator $\underline{Q}$ (resp. $\overline{Q}$) on $\smash{\mathbb{R}^{\mathcal{X}}}$ is a lower (resp. upper) rate operator if for all $f,g\in\smash{\mathbb{R}^{\mathcal{X}}}$, all $\lambda\in\mathbb{R}_{\geq 0}$ and $\mu\in\mathbb{R}$, and all $x,y\in\mathcal{X}$ with $y\neq x$, it holds that [De Bock, 2017]

1. 1.

   $\underline{Q}(\mu\mathbf{1})(x)=0$ and $\overline{Q}(\mu\mathbf{1})(x)=0$

2. 2.

   $\underline{Q}\mathbb{I}_{y}(x)\geq 0$ and $\overline{Q}\mathbb{I}_{y}(x)\geq 0$

3. 3.

   $\underline{Q}f+\underline{Q}g\leq\underline{Q}(f+g)$ and $\overline{Q}(f+g)\leq\overline{Q}f+\overline{Q}g$

4. 4.

   $\underline{Q}(\lambda f)=\lambda\underline{Q}f$ and $\overline{Q}(\lambda f)=\lambda\overline{Q}f$

As before, such objects are conjugate, in that if $\underline{Q}$ is a lower rate operator, then $\smash{\overline{Q}}(\cdot)=-\smash{\underline{Q}}(-\cdot)$ is an upper rate operator. Moreover, any rate matrix $Q$ is also a lower (and upper) rate operator. There is again a duality between lower (or upper) rate operators, and sets of rate matrices. For fixed $\underline{Q}$ and with the dominating set of rate matrices $\mathcal{Q}_{\underline{Q}}$ defined as

we have the following result:

Now fix any lower rate operator $\underline{Q}$ and any $t\in\mathbb{R}_{\geq 0}$, and let

The operator $e^{\underline{Q}t}$ is then a lower transition operator [De Bock, 2017], and the family $(e^{\underline{Q}t})_{t\in\mathbb{R}_{\geq 0}}$ is a semigroup of lower transition operators; it satisfies $e^{\underline{Q}(t+s)}=e^{\underline{Q}t}e^{\underline{Q}s}$ for all $t,s\in\mathbb{R}_{\geq 0}$, and $e^{\underline{Q}0}=I$. The analogous construction with an upper rate operator $\overline{Q}$ instead generates a semigroup $(e^{\overline{Q}t})_{t\in\mathbb{R}_{\geq 0}}$ of upper transition operators. When $\underline{Q}$ and $\overline{Q}$ are taken with respect to the same set $\mathcal{Q}$, these semigroups satisfy, for all $t\in\mathbb{R}_{\geq 0}$, $f\in\smash{\mathbb{R}^{\mathcal{X}}}$, and $Q\in\mathcal{Q}$,

Here the importance again derives from the use as a computational tool; under the conditions of duality between $\mathcal{Q}$ and $\underline{Q}$, we have for any $f\in\smash{\mathbb{R}^{\mathcal{X}}}$ and any $t\in\mathbb{R}_{\geq 0}$ that [Škulj, 2015, Krak et al., 2017]

Hence such inferences can be numerically computed by approximating (6) with a finite choice of $n$, and then solving $n$ independent linear optimization problems over $\mathcal{Q}$. Error bounds for this scheme are available in the literature [Škulj, 2015, Krak et al., 2017, Erreygers, 2021].

Let us now fix a set $\mathcal{Q}$ of rate matrices that we will use in the remainder of this work. Throughout, let $\underline{Q}$ and $\overline{Q}$ denote the lower- and upper rate operators associated with $\mathcal{Q}$. We impose several standard regularity conditions on this set: we assume that $\mathcal{Q}$ is non-empty, compact, convex, and that it has separately specified rows. These are common assumptions that are imposed to ensure the duality between $\mathcal{Q}$ and $\underline{Q}$, which in turn guarantees that inferences with the induced imprecise-Markov chains remain well-behaved, as well as analytically (and, often, computationally) tractable.

We now have all the pieces to start studying the inference problem that is the subject of this work: the lower- and upper expected hitting times of the set $A\subset\mathcal{X}$ for continuous-time imprecise-Markov chains described by $\mathcal{Q}$.

Before we begin, let us impose two additional conditions on the dynamics of the system.

Note that this does not influence the inferences in which we are interested, since those only deal with behavior at times before states in $A$ are reached. However, imposing this explicitly substantially simplifies the analysis.

Next, we assume that the set $A$ is lower reachable from any state $x\in A^{c}$ [De Bock, 2017]. This means that we can construct a sequence $x_{1},\ldots,x_{n+1}\in\mathcal{X}$ starting in any $x_{1}\in A^{c}$ and ending in some $x_{n+1}\in A$ such that, for all $k=1,\ldots,n$, it holds that $\underline{Q}\mathbb{I}_{x_{k+1}}(x_{k})>0$. This is equivalent [De Bock, 2017] to

Essentially, this means that for all elements of our imprecise-probabilistic models the probability of eventually hitting $A$ is bounded away from zero. This ensures that the expected hitting times remain bounded for all $P\in\mathcal{P}^{\mathrm{I}}_{\mathcal{Q}}$, so that we can ignore any extended real-valued analysis. It also implies that for all $Q\in\mathcal{Q}$ we have that $Q(x,x)\neq 0$ for all $x\in A^{c}$, which is relevant to meet the precondition of Proposition 2. As a practical point, De Bock [2017] gives an algorithm to check whether a given set $\mathcal{Q}$ satisfies this condition.

On a technical level, Assumption 2 is the crucial one for our results, and—unlike with Assumption 1—it cannot really be ignored in practice. However, based on earlier work by Krak et al. [2019] in the discrete-time setting, we hope in the future to strengthen our results to hold without this assumption.