This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

An Optimal Switching Approach for Bird Migration

Jiawei Chua,b,   King-Yeung Lamb, Corresponding author:[email protected]; KYL is supported by National Science Foundation Grant DMS-2325195   Boyu Wangc   and  Tong Wangb
aDepartment of Applied Mathematics, The Hong Kong Polytechnic University,
Kowloon, Hong Kong.
bDepartment of Mathematics, The Ohio State University,
Columbus, 43210, Ohio, USA.
cSchool of Mathematics, Fudan University,
Shanghai 200433, China
Abstract

Bird migration is an adaptive behavior ultimately aiming at optimizing survival and reproductive success. We propose an optimal switching model to study bird migration, where birds’ migration behaviors can be efficiently modeled as switching between different stochastic differential equations. For individuals with perfect information regarding the environment, we implement numeric methods to see the expected payoff and corresponding optimal control. For individual with only partial information of the environment, we combine the finite difference method and stochastic simulations to investigate the change of the bird’s optimal strategy. Based on biological backgrounds, we characterizing the optimal strategies of birds under different scenarios and these behaviors depend on the specific assumptions of the model.

1 Introduction

1.1 Biological setting and general questions

Migration is a seasonal movement of organisms between seasonal favorable regions. Bird migration, in particular, is remarkable for their long distances, geological barriers involved and predation risks. Each year, billions of birds driven by the optimal living habitats take on journeys that span up to thousands of kilometers, across ocean and continents. Avoiding intraspecific competition and exploiting crucial resources in seasonal favorable regions for breeding were usually considered as two main elements for evolution of bird migration [32]. Various bird species show a wide range of complex migratory behaviors. For example, different species show both high and low route repeatability, consistency and variation in duration of migration [35], and different migratory strategies between spring and fall migration [25]. The route and schedule of migration are affected by many factors, including the presence of geological barriers, predation risk [39], global warming [16], access to climate information at the terminal site [7], and population density dependence [12]. Although bird migrations exhibit complex behaviors, they all serve the same purpose: to optimize survival probability and reproductive success. Stochastic optimal control, an effective tool for modeling population dynamics in biological/ecological systems [19, 40], is usually employed to model bird migration [1, 14, 23, 30].

1.2 Stochastic optimization approach

Optimal control theory is ubiquitous in theoretical and applied ecology. It predicts how an individual agent can navigate a set of risks and reward to maximize a payoff functional, which includes the reproductive gain and the survival rate in a migration process. In general, a type of behavior maximizing the payoff functional will tend to increase in frequency in the population if they have a heritable component through genetics or learning. The payoff functional is often stochastic (e.g., depending on the mean arrival time at the destination following a diffusion process), and is expressed in terms of an expectation. In such a case, stochastic dynamic programming (SDP) can be used to reformulate the stochastic optimization problem into a set of deterministic partial differential equations, to be solved backward in time. Here we follow the convention in ecology and resource management to refer to both the model itself and the solution approach as SDP.

SDP has been widely used in population dynamics [19, 40], particularly in the study of bird migration [1, 14, 23, 30]. Conventional bird migration models based on SDP [30] are usually discretized in space, using a large number of variables to characterize the migration process, which often makes them analytically intractable. For continuous-time models, solving a stochastic optimization problem reduces to analyze the underlying Hamilton-Jacobi-Bellman (HJB in short) equation [11, Chapter 9] by using analytical method (e.g.,[42]) or numerical techniques (e.g., [26]), and the optimal feedback control may be obtained as a byproduct.

1.3 Our objective

The objective of this paper is to propose an optimal switching model of bird migration with stopover decision based on SDP. The optimal switching model [29, Chapter 5] is a decision-making framework in which individuals switch between two or more diffusion processes to maximize objective function. In our model, the migratory birds switch between three states: “detour”, “direct flight”, and “resting at a stopover site”, each is governed by a separate stochastic differential equation (see Section 2). The optimal switching strategy in our model (location and timing to switch between states) is derived by studying a deterministic system of HJB variational inequalities through both analytical and numerical framework, see Section 3 and 4, respectively. Finally, in Section 5, we apply this optimal switching model to answer several specific biological questions by characterizing the optimal strategies of birds under different scenarios.

1.4 Applications of optimal control in our model

There are many interesting biological questions arising from animal migration. The first question we discuss in this paper is the effect of the deterioration of stopover site. Climate change can dramatically reshape the migration behaviors of birds [1, 37, 36]. Extensive research has demonstrated the profound influence of temperature increase on bird migration choice [37, 5]. Furthermore, the loss of wetland due to increasing temperature [3] can greatly reduce the number of options of migratory shorebirds which depend heavily on the stopover wetlands along the coast for food supplies. Hence, it is natural to ask that

  • Can we quantify and analyze the change in their migratory payoff as the quality of major stopover sites decline, and predict the corresponding change in the migration strategy?

We shall explore this question and present our simulation results in Subection 5.1.

Another important factor influencing the migration process is the weather dynamics [7]. Instead of focusing on the stopover site, we will place more emphasis on the effect of the stochastic weather condition happening at the terminal site. However, in nature, birds can anticipate environmental change based on past experience and perception [7, 6] but only to a limited degree. In particular, they may not be aware of every spontaneous change of climate, especially the condition at the terminal site when it is far away. To this end, we consider the effect of imperfect versus perfect information about the weather condition at the terminal site. Based on the principle of adaptation, we assume that birds have formed a stable strategy according to the average condition of the weather dynamics at the terminal site, which is based on the past experience. But the actual quality of the terminal site may contain unpredictable daily fluctuation due to stochasticity, or be shifted in time due to global climate change. These factors lead to potential mismatch of the perceptual quality of the terminal site versus its actual quality that the bird experiences that particular year.

As such, we will additionally address these two questions in Subection 5.2:

  • Will only knowing the information about the averaged condition at the terminal site decrease the expected payoff significantly if we compare with the same situation but with perfect information?

  • How will the migratory payoff be impacted if the optimal arrival date (or peak green-timing) deviates significantly from past experience, due to the global climate change [31]?

2 Mathematical modeling

In this section, we formulate the migration dynamics and the objective function to be optimized.

2.1 Basic setting

Refer to caption
Figure 1: Geographical setting of the migration route.

We model the bird migration process as a stochastic optimal switching problem. In this model, individuals switch between different states (representing detour flight, direct flight and waiting) where they are governed by a different drift-diffusion process. A value function VV representing the expected payoff function when optimal control is exercised. To evaluate VV, we derive a system of variational inequalities and prove that VV is the unique viscosity solution of the system of variational inequalities (see Sections 2.4 and 3).

The essential idea of the model can be presented by a simple qualitative example of a migrating Pacific brant, which travels more than 5000 km from breeding ground at Izembek Lagoon on the Alaska Peninsula to their wintering ground in Southern Baja California or mainland Mexico [30]. Each individual has a TT-day period to complete the migration to its terminal site. For simplicity, we consider that the bird can not survive unless it reaches the terminal site.

For the model, we model the Pacific as a linear interval [0,L][0,L] (with x=0x=0 being the starting location, and x=Lx=L being the destination). A typical individual can switch between the two states: “detour flight” or “wait” as it moves along the interval [0,L][0,L], where “wait” is only possible at one of the MM stopover sites. Additionally, as individual departs from one of the MM stopover sites, it may choose to embark on a direct flight across the Pacific ocean towards the terminal site (Baja Peninsula), which is modeled by the state “direct flight” which has a greater mean speed and variability comparing to the “detour flight”. See Figure 1 for an illustration. Instead of using a two-dimensional graph, we model the choice of direct flight by the fact that an individual cannot swtich back to “detour flight” or “wait” once it adopts a “direct flight”, and the higher mean speed also reflects that fact that the physical distance of a direct flight (crossing the ocean) is smaller than a detour flight (along the coast).

The whole switching process is shown in Figure 2. Here we label the ii-th stopover sites by the interval [Liϵ,Li+ϵ][L_{i}-\epsilon,L_{i}+\epsilon]. For x [0,L1ϵ)\in[0,L_{1}-\epsilon), where the bird still migrate within the continent and do not reach the ocean, the individual bird continues in the state “detour flight” until it reaches the first stopover site [L1ϵ,L1+ϵ][L_{1}-\epsilon,L_{1}+\epsilon] (representing the first staging area near the ocean). At this point, it can choose either to detour along the coast or to take direct flight over the ocean. If it adopts a direct flight, then it switches to the respective diffusion process which does not terminate until it arrives at the destination. Otherwise, it continues to the next stopover site and the process repeats itself.

Refer to caption
Figure 2: The three states of an individual (waiting/detour/direct) with switching control

2.2 Migration dynamics

We translate the bird migration into mathematical framework for optimal switching problem. Denote LL as the distance between starting point and terminal site, and T<T<\infty as the terminal time for bird migration, then we consider an optimal switching problem on a finite horizon. Fix a probability space (Ω,,P)(\varOmega,\mathcal{F},P) with a filtration 𝔽=(s)st\mathbb{F}=(\mathcal{F}_{s})_{s\geqslant t} satisfying the usual conditions (see, e.g. [11, 29]). The regime space

Π=Πm={1,2,.,m}\Pi=\Pi_{m}=\{1,2,....,m\}

(with m=3m=3 in our case), and describes the state that a bird is staying in. Then denote (t,x,i)[0,T]×[0,L]×Π(t,x,i)\in[0,T]\times[0,L]\times\Pi as the initial conditions, which describes the exact position xx of an individual bird at time tt and regime ii.

A switching control is a double sequence α=(τn,ιn)n1\alpha=(\tau_{n},\iota_{n})_{n\geqslant 1}, where (τn)(\tau_{n}) are an increasing sequence of stopping times, τn\tau_{n}\rightarrow\infty as nn\rightarrow\infty a.s., denote the decision on “when to switch”; ιn\iota_{n} are τn\mathcal{F}_{\tau_{n}}-measurable valued in Π\Pi, represent the new value of the regime during time [τn,τn+1)[\tau_{n},\tau_{n+1}). The set of all switching controls is denoted as 𝒜\mathcal{A}. Given the initial regime ii, we can define the controlled switching process among different states as follows:

πs:=π(s,i)=n0ιnχ[τn,τn+1)(s),st,πt=i,\pi_{s}:=\pi(s,i)=\sum_{n\geq 0}\iota_{n}\chi_{[\tau_{n},\tau_{n+1})}(s),s\geq t,\quad\pi_{t^{-}}=i, (2.1)

where χ[τn,τn+1)\chi_{[\tau_{n},\tau_{n+1})} is the indicator function, defined by

χB:=χB(y)={1,ifyB,0,otherwise;\chi_{B}:=\chi_{B}(y)=\begin{cases}1,&\text{if}~{}y\in B,\\ 0,&\text{otherwise};\end{cases}

and we set τ0:=t\tau_{0}:=t, ι0:=i\iota_{0}:=i and t[0,T)t\in[0,T) is any fixed. Then we use a Markov process {Xst,x,i}st\{X^{t,x,i}_{s}\}_{s\geq t} to represent the whole migration, starting from position xx and the state ii at an initial time tt, which satisfies the following stochastic differential equation

dXst,x,i=j=13vjχ{j=πs}ds+j=132μjχ{j=πs}dBs,for0ts<T.\begin{split}dX_{s}^{t,x,i}&=\sum_{j=1}^{3}v_{j}\chi_{\{j=\pi_{s}\}}ds+\sum_{j=1}^{3}\sqrt{2\mu_{j}}\chi_{\{j=\pi_{s}\}}dB_{s},\ \ \text{for}~{}0\leq t\leq s<T.\end{split} (2.2)

For the remainder of this article, let the regime state i=1i=1 represent the state of detour flight, i=2i=2 represent the state of direct flight, and i=3i=3 represent the state of waiting, and let BsB_{s} be a 1-dimensional Brownian motion on the filtered probability space (Ω,,𝔽,P)(\varOmega,\mathcal{F},\mathbb{F},P) satisfying the usual conditions. The constants vj0v_{j}\geq 0 and μj0(j=1,2,3)\mu_{j}\geq 0~{}(j=1,2,3) represent the velocity and volatility of the regime, respectively, where viv_{i} and μi\mu_{i} are constants satisfying

v2>v1>0=v3 and μ1>μ2>0=μ3,v_{2}>v_{1}>0=v_{3}\quad\text{ and }\quad\mu_{1}>\mu_{2}>0=\mu_{3},

which is motivated by the fact that directly flight shortens the route distance to destination (v2>v1v_{2}>v_{1}) while increases stochasticity (μ2>μ1\mu_{2}>\mu_{1}) owing to weather variations, such as wind direction, wind velocity, etc.

Consequently, for any initial condition (t,x,i)(t,x,i) and any given control α𝒜\alpha\in\mathcal{A}, the migration dynamics can alternative be written as:

dXst,x,i=vιnds+2μιndBs,πs=ιn,s[τn,τn+1),n1,dX^{t,x,i}_{s}=v_{\iota_{n}}ds+\sqrt{2\mu_{\iota_{n}}}dB_{s},\ \ \pi_{s}=\iota_{n},\forall~{}s\in[\tau_{n},\tau_{n+1}),\ \ n\geq 1, (2.3)

where πs\pi_{s} is defined in (2.1). When ιn3\iota_{n}\neq 3, from [27, Theorem V.38], we get that there exists a unique strong solution valued in [0,L][0,L] to the standard SDE (2.3) for any (t,x)[0,T]×[0,L](t,x)\in[0,T]\times[0,L], denote it by Xst,x,iX^{t,x,i}_{s}; when ιn=3\iota_{n}=3, we set, for each (t,x)[0,T]×[0,L](t,x)\in[0,T]\times[0,L],

Xst,x,i=Xτnt,x,i for all s[τn,τn+1].X^{t,x,i}_{s}=X^{t,x,i}_{\tau_{n}}\quad\text{ for all }s\in[\tau_{n},\tau_{n+1}]. (2.4)

2.3 Objective function

The objective function is a function to be maximized by the individual bird through its migration process subject to the migration dynamics formulated in the previous subsection. Here are some key components for the construction of objective function.

  • (H1)

    Running and terminal reward functions. For j=1,2,3j=1,2,3, we let fj(t,x)=f(t,x,j):[0,T]×[0,L]×Πf_{j}(t,x)=f(t,x,j):~{}[0,T]\times[0,L]\times\Pi\rightarrow\mathbb{R} and gj(t,x)=g(t,x,j):g_{j}(t,x)=g(t,x,j): [0,T]×[0,L]×Π[0,T]\times[0,L]\times\Pi\rightarrow\mathbb{R} be, respectively, the running and terminal reward functions. We assume that they are uniformly Lipschitz continuous in (t,x)(t,x).

  • (H2)

    Mortality/Discount rate. For j=1,2,3j=1,2,3, we denote the mortality rate associated with birds’ action by βj(t,x)=β(t,x,j):[0,T]×[0,L]×Π\beta_{j}(t,x)=\beta(t,x,j):~{}[0,T]\times[0,L]\times\Pi\rightarrow\mathbb{R} and assume that it is Lipschitz continuous. In reality, the mortality rate for direct flight is considered higher than that for detour flight, reflecting the increased challenges birds face over the sea, such as the inability to rest periodically and greater exposure to adverse weather conditions.

  • (H3)

    Switching cost. We denote the cost to switch from regime kk to jj, by hkj(t,x):[0,T]×[0,L]h_{kj}(t,x):~{}[0,T]\times[0,L]\rightarrow\mathbb{R} for k,j{1,2,3}k,j\in\{1,2,3\}. For each k,jk,j, we assume 0hkj(t,x)C1,2([0,T]×[0,L])0\leq h_{kj}(t,x)\in C^{1,2}([0,T]\times[0,L]), where the equality holds iff k=jk=j. Moreover, suppose that the function hkj(t,x)h_{kj}(t,x) satisfy

    hkj<hkq+hqj, for qk,j, such that hkq+hqj<+,h_{kj}<h_{kq}+h_{qj},\ \ \text{ for }q\neq k,~{}j,\text{ such that }h_{kq}+h_{qj}<+\infty, (2.5)

    and

    h2j=+,j=1,3.h_{2j}=+\infty,\ \ j=1,~{}3. (2.6)

    The condition (2.5) means that switching in two steps via an intermediate regime qq is more costly than in one step from regime kk to jj; (2.6) implies that the bird cannot switch from direct flight to detour or waiting state.

The arrival time at x=Lx=L is a hitting time of the process:

τ=inf{st:Xst,x,i=L}.\tau=\inf\{s\geq t:X^{t,x,i}_{s}=L\}.

The objective function give the expected benefit, given the current state (t,x,i)[0,T]×[0,L]×Π(t,x,i)\in[0,T]\times[0,L]\times{\Pi} and subject to a given control α=(τn,ιn)n1𝒜\alpha=(\tau_{n},\iota_{n})_{n\geq 1}\in\mathcal{A}:

Ji(t,x,α)=Et,x,i[tτTeβπs(s,Xst,x,i)(st)fπs(s,Xst,x,i)dstτn<τTeβπs(τn,Xst,x,i)(τnt)hln1ln(s,Xst,x,i)+eβπs(τ,Xτt,x,i)(τt)gπs(τ,L)χ{τT}],\begin{split}J_{i}(t,x,\alpha)=E^{t,x,i}\bigg{[}\int_{t}^{\tau\wedge T}&e^{-\beta_{\pi_{s}}(s,X_{s}^{t,x,i})(s-t)}f_{\pi_{s}}(s,X_{s}^{t,x,i})ds-\sum_{t\leq\tau_{n}<\tau\wedge T}e^{-\beta_{\pi_{s}}(\tau_{n},X_{s}^{t,x,i})(\tau_{n}-t)}{h_{l_{n-1}l_{n}}}(s,X_{s}^{t,x,i})\\ &+e^{-\beta_{\pi_{s}}(\tau,X_{\tau}^{t,x,i})(\tau-t)}g_{\pi_{s}}(\tau,L)\chi_{\{\tau\leq T\}}\bigg{]},\end{split} (2.7)

where Xst,x,iX_{s}^{t,x,i} is the unique strong solution to (2.2) subject to the given control process α\alpha. Under proper assumptions (see (H1)-(H3)) Ji(t,x,α)J_{i}(t,x,\alpha) is well-defined for any initial condition (t,x,i)(t,x,i) and any control process α𝒜\alpha\in\mathcal{A}.

Finally, the value function of the optimal control problem is defined as the supremum of the objective function Ji(t,x,α)J_{i}(t,x,\alpha) over the set of all admissible control processes 𝒜\mathcal{A}:

Vi(t,x)=supα𝒜Ji(t,x,α),(t,x,i)[0,T]×[0,L]×Π.\begin{split}V_{i}(t,x)=\sup_{\alpha\in\mathcal{A}}J_{i}(t,x,\alpha),\ \ {(t,x,i)\in[0,T]\times[0,L]\times\Pi.}\end{split} (2.8)

By (2.7), it is natural to set the terminal condition

Vi(T,x)=0 for x[0,L),V_{i}(T,x)=0\quad\text{ for }x\in[0,L),\\

and boundary conditions

(Vi)x(t,0)=0,Vi(t,L)=gi(t,L), for t[0,T],(V_{i})_{x}(t,0)=0,\quad V_{i}(t,L)=g_{i}(t,L),\quad\text{ for }t\in[0,T],

which says that the diffusion process is reflected at the boundary x=0x=0 and is absorbing at x=Lx=L.

The goal of this optimal switching problem is to find a strategy α=(τn,ιn)n1𝒜\alpha^{*}=(\tau^{*}_{n},\iota^{*}_{n})_{n\geq 1}\in\mathcal{A} to achieve the maximum of (2.8), α\alpha^{*} is called the optimal strategy. A key ingredient of the Bellman’s approach of stochastic optimal control is the dynamic programming principle, which we now recall.

Lemma 2.1 (Stochastic dynamic programming principle (SDPP)).

For any (t,x,i)[0,T]×[0,L]×Π(t,x,i)\in[0,T]\times[0,L]\times\Pi, we have

Vi(t,x)=supα𝒜Et,x,i[tτθ\displaystyle V_{i}(t,x)=\sup_{\alpha\in\mathcal{A}}E^{t,x,i}\bigg{[}\int_{t}^{\tau\wedge\theta} eβπs(s,Xst,x,i)(st)f(s,Xst,x,i,πs)dstτn<τθeβπs(τn,Xst,x,i)(τnt)hln1ln(s,Xst,x,i)\displaystyle e^{-\beta_{\pi_{s}}(s,X_{s}^{t,x,i})(s-t)}f(s,X_{s}^{t,x,i},\pi_{s})ds-\sum_{t\leq\tau_{n}<\tau\wedge\theta}e^{-\beta_{\pi_{s}}(\tau_{n},X_{s}^{t,x,i})(\tau_{n}-t)}h_{l_{n-1}l_{n}}(s,X_{s}^{t,x,i})
+eβπs(τ,Xst,x,i)(τt)gi(τ,L)χ{τ<θ}+eβπs(θ,Xst,x,i)(θt)Vi(θ,Xθt,x,i)χ{θτT}],\displaystyle+e^{-\beta_{\pi_{s}}(\tau,X_{s}^{t,x,i})(\tau-t)}g_{i}(\tau,L)\chi_{\{\tau<\theta\}}+e^{-\beta_{\pi_{s}}(\theta,X_{s}^{t,x,i})(\theta-t)}V_{i}(\theta,X_{\theta}^{t,x,i})\chi_{\{\theta\leq{\tau\wedge T}\}}\bigg{]},

where θ[t,T]\theta\in[t,T] is any stopping time, possibly depending on α𝒜\alpha\in\mathcal{A}.

Proof.

See, e.g. [11, Lemma 4.2]. ∎

2.4 Hamilton-Jacobi-Bellman variational inequality

To find the optimal control α\alpha^{*}, the crucial step is to show, based on the the dynamic programming principle (see Lemma 2.1) and the concept of viscosity solutions (see Definition 3.1), that (Vi(t,x))iΠ(V_{i}(t,x))_{i\in\Pi} is the unique viscosity solution to the following system of Hamilton-Jacobi-Bellman (HJB in short) variational inequality with terminal and lateral conditions:

{min[tVi+βiViiVifi,Vimaxij(Vjhij)]=0,(t,x)(0,T)×(0,L),(Vi)x(t,0)=0,t[0,T],Vi(t,L)=gi(t,L),t[0,T],Vi(T,x)=0,x[0,L),\begin{cases}\min[-\partial_{t}V_{i}+\beta_{i}V_{i}-\mathcal{L}_{i}V_{i}-f_{i},V_{i}-\max_{i\neq j}(V_{j}-h_{ij})]=0,&(t,x)\in(0,T)\times(0,L),\\ {(V_{i})_{x}}(t,0)=0,&t\in[0,T],\\ V_{i}(t,L)=g_{i}(t,L),&t\in[0,T],\\ V_{i}(T,x)=0,&x\in[0,L),\end{cases} (2.9)

where i{1,2,3}i\in\{1,2,3\} and

iϕ:=viϕx+μiϕxx\mathcal{L}_{i}\phi:=v_{i}\phi_{x}+\mu_{i}\phi_{xx} (2.10)

(see details in Proposition 3.2 in Section 3).

This system of PDE facilitates the identification of the associated optimal control α\alpha^{*}. Precisely, for any iji\neq j, we define the switching region to be the closed set

𝒮ij={(t,x)(0,T)×(0,L):Vi(t,x)=(Vjhij)(t,x)},\mathcal{S}_{ij}=\{(t,x)\in(0,T)\times(0,L):V_{i}(t,x)=(V_{j}-h_{ij})(t,x)\}, (2.11)

which means that if an individual in state ii is located at location xx and time tt, such that (t,x)𝒮ij(t,x)\in\mathcal{S}_{ij}, then it will switch from regime ii to regime jj. Define

𝒮i=ji𝒮ij and 𝒞i=[0,T]×[0,L]𝒮i.\mathcal{S}_{i}=\cup_{j\neq i}\mathcal{S}_{ij}\quad\text{ and }\quad\mathcal{C}_{i}=[0,T]\times[0,L]\setminus\mathcal{S}_{i}. (2.12)

It follows from [28, Lemma 4.2] that

𝒮i={(t,x)(0,T)×(0,L):Vi(t,x)=maxji(Vjhij)(t,x)},\mathcal{S}_{i}=\{(t,x)\in(0,T)\times(0,L):V_{i}(t,x)=\max_{j\neq i}(V_{j}-h_{ij})(t,x)\},

then

𝒞i={(t,x)[0,T]×[0,L]:Vi(t,x)>maxji(Vjhij)(t,x)}.\mathcal{C}_{i}=\{(t,x)\in[0,T]\times[0,L]:V_{i}(t,x)>\max_{j\neq i}(V_{j}-h_{ij})(t,x)\}.

By Proposition 3.3, we get that 𝒮i\mathcal{S}_{i} is the region where changing the regime ii is optimal. If (t,x)𝒞i(t,x)\in\mathcal{C}_{i}, then it is optimal to stay in regime ii at least for a short time (Proposition 3.3). Hence, 𝒞i\mathcal{C}_{i} is called the continuation region.

Moreover, by (2.5), the switching regions are disjoint, so that

𝒮ij𝒞j,jiΠ.\mathcal{S}_{ij}\subset\mathcal{C}_{j},\ \ j\neq i\in\Pi.

Roughly speaking, an individual who has just switched from regime ii to jj does switch immediately from jj to another regime. When i=2i=2, assumption (H3)implies that there is no switch, this gives 𝒮2=\mathcal{S}_{2}=\emptyset and 𝒞2=[0,T]×[0,L]\mathcal{C}_{2}=[0,T]\times[0,L].

Consequently, the optimal strategy for an individual bird/agent can be fully characterized by the switching regions 𝒮12,𝒮13,𝒮31,𝒮32\mathcal{S}_{12},\mathcal{S}_{13},\mathcal{S}_{31},\mathcal{S}_{32}, as given in (2.11).

3 Theory

In this section, we first give the definition of vicosity solution, and then we shall show that (Vi(t,x))iΠ(V_{i}(t,x))_{i\in\Pi} is the unique viscosity solution to (2.9)(see Propostion 3.2). This result enables us to get the associated optimal control α\alpha^{*} (see Proposition 3.3).

Let t[0,T]t\in[0,T] and 𝒪n(n1)\mathcal{O}\subset\mathbb{R}^{n}~{}(n\geq 1) be an open connected domain with closure 𝒪¯\overline{\mathcal{O}} and smooth boundary 𝒪:=𝒪¯\𝒪\partial\mathcal{O}:=\overline{\mathcal{O}}\verb|\|\mathcal{O} satisfying the exterior ball condition. Let 𝒪=Γ1Γ2\partial\mathcal{O}=\Gamma_{1}\cup\Gamma_{2} with Γ1\Gamma_{1} being open set and Γ2\Gamma_{2} being closed set. Consider the following general system of HJB variational inequality with mixed boundary conditions:

{min[tui+Fi(t,x,ui,Dui,D2ui),uimaxij(ujhij)]=0,(t,x)(0,T)×𝒪,νui(t,x)=ϕi(t,x),(t,x)(0,T)×Γ1,ui(t,x)=di(t,x),(t,x)[0,T)×Γ2,ui(T,x)=ψi(x),x𝒪¯,\begin{cases}\min[-\partial_{t}u_{i}+F_{i}(t,x,u_{i},Du_{i},D^{2}u_{i}),u_{i}-\max_{i\neq j}(u_{j}-h_{ij})]=0,&(t,x)\in(0,T)\times\mathcal{O},\\ \partial_{\nu}u_{i}(t,x)=\phi_{i}(t,x),&(t,x)\in(0,T)\times\Gamma_{1},\\ u_{i}(t,x)=d_{i}(t,x),&(t,x)\in[0,T)\times\Gamma_{2},\\ u_{i}(T,x)=\psi_{i}(x),&x\in\overline{\mathcal{O}},\end{cases} (3.13)

where Fi(t,x,ui,Dui,D2ui):=βiuiiuifiF_{i}(t,x,u_{i},Du_{i},D^{2}u_{i}):=\beta_{i}u_{i}-\mathcal{L}_{i}u_{i}-f_{i} with i\mathcal{L}_{i} defined in (2.10) and νui:=uiν\partial_{\nu}u_{i}:=\frac{\partial u_{i}}{\partial\nu} with ν\nu denoting the unit outward normal vector of boundary Γ1\Gamma_{1}. We suppose that

  • (Ha)

    Nonnegative functions βj(t,x)\beta_{j}(t,x) and fj(t,x)f_{j}(t,x): [0,T]×𝒪¯×Π[0,T]\times\overline{\mathcal{O}}\times\Pi\rightarrow\mathbb{R} are uniformly Lipschitz continuous in (t,x)(t,x); the switching cost 0hkj(t,x)C1,2([0,T]×𝒪¯):[0,T]×𝒪¯0\leq h_{kj}(t,x)\in C^{1,2}([0,T]\times\overline{\mathcal{O}}):~{}[0,T]\times\overline{\mathcal{O}}\rightarrow\mathbb{R} for k,jΠk,j\in\Pi satisfies (2.5), where “=” holds iff k=jk=j.

  • (Hb)

    For each iΠi\in\Pi, ϕi(t,x)\phi_{i}(t,x) and di(t,x)d_{i}(t,x) are continuous on [0,T]×Γ1[0,T]\times\Gamma_{1} and [0,T]×Γ2[0,T]\times\Gamma_{2}, respectively. The function ψi(x):𝒪¯×Π\psi_{i}(x):\overline{\mathcal{O}}\times\Pi\rightarrow\mathbb{R} is continuous such that

    ψi(x)maxij(ψj(x)hij(T,x)).\psi_{i}(x)\geq\max_{i\neq j}(\psi_{j}(x)-h_{ij}(T,x)).

Under assumptions (Ha)-(Hb), we can introduce the concept of viscosity solutions. We first recall some fundamental notations. Given a locally bounded function u(t,x):[0,T]×𝒪¯u(t,x):[0,T]\times\overline{\mathcal{O}}\rightarrow~{}\mathbb{R} (i.e., for all (t,x)[0,T]×𝒪¯(t,x)\in[0,T]\times\overline{\mathcal{O}}, there exists a compact neighborhood V(t,x)V_{(t,x)} of (t,x)(t,x) such that uu is bounded on V(t,x)V_{(t,x)}). We recall that its lower-semicontinuous envelope uu_{*} and upper-semicontinuous envelope uu^{*} on [0,T]×𝒪¯[0,T]\times\overline{\mathcal{O}} (see [11, pp.267, Definition 4.1]) are given respectively by

u(t,x):=lim inf(s,y)(t,x)u(s,y):=supr>0inf{u(s,y)[0,T]×𝒪¯Br(t,x)},u(t,x):=lim sup(s,y)(t,x)u(s,y):=infr>0sup{u(s,y)[0,T]×𝒪¯Br(t,x)},\begin{split}&u_{*}(t,x):=\liminf_{(s,y)\rightarrow(t,x)}u(s,y):=\sup\limits_{r>0}\text{inf}\{u(s,y)\in[0,T]\times\overline{\mathcal{O}}\cap B_{r}(t,x)\},\\ &u^{*}(t,x):=\limsup_{(s,y)\rightarrow(t,x)}u(s,y):=\inf\limits_{r>0}\text{sup}\{u(s,y)\in[0,T]\times\overline{\mathcal{O}}\cap B_{r}(t,x)\},\end{split}

i.e., uu_{*} (resp. uu^{*}) is the largest (resp. smallest) lower-semicontinuous function (l.s.c.) below (resp. upper-semicontinuous function (u.s.c.) above) uu on [0,T]×𝒪[0,T]\times\mathcal{O}. Observe that u(s,y)u(s,y) is continuous at (t,x)[0,T]×𝒪(t,x)\in[0,T]\times\mathcal{O} if and only if u(t,x)=u(t,x)=u(t,x)u(t,x)=u_{*}(t,x)=u^{*}(t,x)

Next, we give the definition of viscosity solution as Definition 4.2 and Remark 4.2 in [11, pp.267] or Definition 4.2.1 in [29], which is equivalent to [22, Definition 2.3] or [4, Definition 3.3] by applying Lemma 4.1 in [11, pp.211].

Definition 3.1 (Viscosity solution).

Let ui(t,x)u_{i}(t,x) (iΠi\in\Pi) be locally bounded. Then function (ui)iΠ(u_{i})_{i\in\Pi} is a viscosity subsolution of (3.13), if for all iΠi\in\Pi and any WC1,2([0,T]×𝒪¯)W\in C^{1,2}([0,T]\times\overline{\mathcal{O}}),

  • (i)

    ui(t,x)W(t,x)u_{i}^{*}(t,x)-W(t,x) admits a maximum point (t¯,x¯)[0,T)×𝒪¯(\bar{t},\bar{x})\in[0,T)\times\overline{\mathcal{O}}, and

    min[tW+Fi(t,x,ui,DW,D2W),uimaxij(ujhij)]0\min[-\partial_{t}W+F_{i}(t,x,u_{i}^{*},DW,D^{2}W),u_{i}^{*}-\max_{i\neq j}(u_{j}^{*}-h_{ij})]\leq 0

    holds for all (t¯,x¯)(\bar{t},\bar{x});

  • (ii-1)

    ui(t,x)W(t,x)u_{i}^{*}(t,x)-W(t,x) admits a maximum point (t¯,x¯)[0,T)×Γ1(\bar{t},\bar{x})\in[0,T)\times\Gamma_{1} and

    min[tW+Fi(t,x,ui,DW,D2W),uimaxij(ujhij)][νuiϕi)]0\min[-\partial_{t}W+F_{i}(t,x,u_{i}^{*},DW,D^{2}W),u_{i}^{*}-\max_{i\neq j}(u_{j}^{*}-h_{ij})]\wedge[\partial_{\nu}u_{i}^{*}-\phi_{i})]\leq 0

    holds for all (t¯,x¯)(\bar{t},\bar{x});

  • (ii-2)
    ui(t,x)di(t,x)0u_{i}^{*}(t,x)-d_{i}(t,x)\leq 0

    holds for all (t,x)[0,T)×Γ2(t,x)\in[0,T)\times\Gamma_{2};

  • (iii)
    ui(T,x)ψi(x)u_{i}^{*}(T,x)\leq\psi_{i}(x)

    holds for all x𝒪¯x\in\overline{\mathcal{O}};

A viscosity supersolutions are defined analogously by replacing (uku_{k}^{*}, \leq, \wedge) with ((uk)(u_{k})_{*}, \geq, \vee), respectively, where ab=min{a,b}a\wedge b=\min\{a,b\} and ab=max{a,b}a\vee b=\max\{a,b\}. A function is a viscosity solution of (3.13) if it is both a viscosity subsolution and viscosity supersolution (3.13).

The comparison principle stated below is derived from [22, Theorem 2.4]; see also the discussion in [29, pp.75].

Lemma 3.1 (Comparison principle).

Let t[0,T]t\in[0,T] and 𝒪n(n1)\mathcal{O}\subset\mathbb{R}^{n}~{}(n\geq 1) be an open connected set with closure 𝒪¯\overline{\mathcal{O}}, and smooth boundary 𝒪:=𝒪¯\𝒪\partial\mathcal{O}:=\overline{\mathcal{O}}\verb|\|\mathcal{O} satisfying the exterior ball condition. Let 𝒪=Γ1Γ2\partial\mathcal{O}=\Gamma_{1}\cup\Gamma_{2} with Γ1\Gamma_{1} being open set and Γ2\Gamma_{2} being closed set. Assume that (Ha)-(Hb) hold. If (ui)iΠ(u_{i})_{i\in\Pi} and (Ui)iΠ(U_{i})_{i\in\Pi} are respectively a visosity subsolution and viscosity supersolution of (3.13), then for each iΠ{i\in\Pi},

uiUi,(t,x)[0,T)×𝒪¯.u_{i}\leq U_{i},\ \ \ \forall~{}(t,x)\in[0,T)\times\overline{\mathcal{O}}.

Using Definition 3.1 and Lemma 3.1, we shall show that (Vi(t,x))iΠ(V_{i}(t,x))_{i\in\Pi} is the unique viscosity solution to (2.9).

Proposition 3.2 (Existence and uniqueness).

For each iΠi\in\Pi, the value function ViC([0,T)×[0,L))V_{i}\in C([0,T)\times[0,L)) is the unique viscosity solution to (2.9) under the Definition 3.1.

Proof.

Take 𝒪=(0,L)\mathcal{O}=(0,L), Γ1=0\Gamma_{1}={0}, Γ2=1\Gamma_{2}={1}, ϕi(x)=0\phi_{i}(x)=0, di(t,x)=gi(t,L)d_{i}(t,x)=g_{i}(t,L) and ψi(x)=0\psi_{i}(x)=0, assumptions (H1)-(H3) indicate that (Ha)-(Hb) hold. Proceeding with the same procedures as the proof in [17, Theorem 5.2] (or see e.g., [29, Theorem 5.3.2]) and [4, Theorem 3.2], we can show that for each i{1,2,3}i\in\{1,2,3\}, the value function ViV_{i} is the viscosity solution to (2.9)under the Definition 3.1.

Next, we prove that (Vi)iΠ(V_{i})_{i\in\Pi} is the unique viscosity solution. We will show the result by contradiction. Assume on the contrary that (Ui)iΠ(U_{i})_{i\in\Pi} and (Vi)iΠ(V_{i})_{i\in\Pi} are both the viscosity solutions to (2.9).Then by definitions of viscosity solution and lower/upper-semicontinuous envelopes, UiU_{i}^{*} and ViV_{i}^{*} (resp. (Ui)(U_{i})_{*} and (Vi)(V_{i})_{*}) are the viscosity subsolutions (resp. viscosity supersolutions). Hence, with assumptions (H1)-(H3) in hand, Lemma 3.1 implies that Ui(Vi)U_{i}^{*}\leq(V_{i})_{*} and Vi(Ui)V_{i}^{*}\leq(U_{i})_{*} on [0,T)×[0,L)[0,T)\times[0,L), which together with the facts (Ui)Ui(U_{i})_{*}\leq U_{i}^{*} and (Vi)Vi(V_{i})_{*}\leq V_{i}^{*} (which hold by construction), gives that

Ui=(Ui)=Ui=(Vi)=Vi=Viin[0,T)×[0,L).U_{i}=(U_{i})_{*}=U_{i}^{*}=(V_{i})_{*}=V_{i}^{*}=V_{i}\ \ \text{in}\ \ [0,T)\times[0,L).

This means that ViC([0,T)×[0,L))V_{i}\in C([0,T)\times[0,L)) is the unique viscosity solution to (2.9) under the Definition 3.1. ∎

The assumption (H3) indicates that no switching occurs if the initial regime state i=2i=2. When the initial regime state i2i\neq 2, [10] and [21] give the following result regarding the optimal strategy.

Proposition 3.3 (Optimal switching strategy).

Let assumptions (H1)- (H3) hold and the initial regime state i{1,3}i\in\{1,3\}. Definie the sequence 𝔽\mathbb{F}-stopping times (τn)n1(\tau^{*}_{n})_{n\geq 1} as follows:

τ1:=inf{st:Vi=maxji(Vjhij)}τT,τn{=inf{sτ1:Vπτn1=maxkτn1(Vkhπτn1k)}τT,ifπτn12;,ifπτn1=2,for anyn2,\begin{split}&\tau^{*}_{1}:=\inf\{s\geq t:V_{i}=\max_{j\neq i}(V_{j}-h_{ij})\}\wedge\tau\wedge T,\\ &\tau^{*}_{n}\begin{cases}=\inf\{s\geq\tau^{*}_{1}:V_{\pi_{\tau^{*}_{n-1}}}=\max\limits_{k\neq\tau^{*}_{n-1}}(V_{k}-h_{\pi_{\tau^{*}_{n-1}}k})\}\wedge\tau\wedge T,&\text{if}\ \ \pi_{\tau^{*}_{n-1}}\neq 2;\\ \nexists,&\text{if}\ \ \pi_{\tau^{*}_{n-1}}=2,\end{cases}\ \ \text{for~{}any}~{}n\geq 2,\end{split}

where,

  • πτ1=kΠkχ{maxji(Vjhij)(τ1,Xτ1t,x,i)=(Vkhik)(τ1,Xτ1t,x,i)}\pi_{\tau_{1}^{*}}=\sum\limits_{k\in\Pi}k\chi_{\{}\max\limits_{j\neq i}(V_{j}-h_{ij})(\tau_{1}^{*},X_{\tau^{*}_{1}}^{t,x,i})=(V_{k}-h_{ik})(\tau_{1}^{*},X_{\tau^{*}_{1}}^{t,x,i})\}

  • for any n1n\geq 1, tτnt\geq\tau_{n}^{*}, Vπτn(t,x)=kΠVk(t,x)χ{πτn=k}V_{\pi_{\tau_{n}^{*}}}(t,x)=\sum\limits_{k\in\Pi}V_{k}(t,x)\chi_{\{\pi_{\tau_{n}^{*}}=k\}};

  • for any n2n\geq 2 and πτn12\pi_{\tau_{n-1}^{*}}\neq 2, πτn=l\pi_{\tau_{n}^{*}}=l on the set

    {maxjπτn1(Vjhπτn1j)(τn,Xτnt,x,i)=(Vlhπτn1l)(τn,Xτnt,x,i)}\Bigg{\{}\max\limits_{j\neq\pi_{\tau_{n-1}^{*}}}(V_{j}-h_{\pi_{\tau_{n-1}^{*}}j})(\tau_{n}^{*},X_{\tau^{*}_{n}}^{t,x,i})=(V_{l}-h_{\pi_{\tau_{n-1}^{*}}l})(\tau_{n}^{*},X_{\tau^{*}_{n}}^{t,x,i})\Bigg{\}}

    with hπτn1j(τn,Xτnt,x,i)=kΠhkj(τn,Xτnt,x,i)χ{πτn1=k}.h_{\pi_{\tau_{n-1}^{*}}j}(\tau_{n}^{*},X_{\tau^{*}_{n}}^{t,x,i})=\sum\limits_{k\in\Pi}h_{kj}(\tau_{n}^{*},X_{\tau^{*}_{n}}^{t,x,i})\chi_{\{\pi_{\tau_{n-1}^{*}}=k\}}.

Then the strategy α=(τn,ιn)n1\alpha^{*}=(\tau^{*}_{n},\iota^{*}_{n})_{n\geq 1} is optimal, i.e., Ji(t,x,α)Ji(t,x,α)J_{i}(t,x,\alpha^{*})\geq J_{i}(t,x,\alpha) for all α𝒜\alpha\in\mathcal{A}.

4 Numerical Methods

This section devoted to the numerical simulation results. To go beyond the standard optimal control framework, we explore the effect of partial vs perfect information in terms of the terminal reward function, representing the payoff an individual received if it survives the trip and arrives at the destination at a certain time tt. For simplicity, we focus on the case where the terminal reward is independent of the regime state (detour vs direct flight on arrival at x=Lx=L). This reward function typically depends on climatic and biotic conditions such as availability of resources and competition. We assume it is a given function of tt for simplicity.

Let g(t)g(t) be the perceived terminal reward as a function of arrival time tt at x=Lx=L, which the individual leverages to optimize its switching strategy. This perceived terminal reward represents the “best guess” of the individual concerning the condition at the terminal site x=Lx=L, which is inhabited from the past experience of the collective experience. Next, let G(t)G(t) be the actual terminal reward. Consider the following two cases

  • (Perfect information)  g(t)=G(t)g(t)=G(t);

  • (Partial information)g(t)G(t).g(t)\neq G(t).

For the case of perfect information, we will use the variational PDE to calculate the value function V(t,x)V(t,x) and obtain the optimal control given by switching regions, and the migratory payoff given the optimal control for individuals started from (0,0)(0,0), which will be represented by V(0,0)V(0,0). This is the subject of the next subsection.

4.1 Perfect information

In Sections 2 and 3, we derived the deterministic PDEs of the value functions for the optimal control problem (2.8)\eqref{eq: optimal_control}. For clarity, we recall that:

{min[tVi+βViiVifi,Vimaxji(Vjhij)]=0(t,x)[0,T)×(0,L),(Vi)x(t,0)=0t[0,T],Vi(t,L)=G(t)t(0,T],Vi(T,x)=0x[0,L),\left\{\begin{aligned} &\min[-\partial_{t}V_{i}+\beta V_{i}-\mathcal{L}_{i}V_{i}-f_{i},V_{i}-\max_{j\neq i}(V_{j}-h_{ij})]=0&(t,x)\in[0,T)\times(0,L),\\ &{(V_{i})_{x}}(t,0)=0&t\in[0,T],\\ &V_{i}(t,L)=G(t)&t\in(0,T],\\ &V_{i}(T,x)=0&x\in[0,L),\end{aligned}\right. (4.14)

where i\mathcal{L}_{i} is given in (2.10). Then, we use a finite difference scheme to solve the coupled system (4.14). We give a brief introduction to the process below.

  • Step 1: The (t,x)(t,x) domain is discretized into a lattice with step sizes Δt\Delta t and Δx\Delta x. Denote Vi,m,n=Vi(mΔt,nΔx)V_{i,m,n}=V_{i}(m\Delta t,n\Delta x), then the parabolic part of the equation can be discretized according to the implicit Euler scheme:

    \displaystyle- Vi,m+1,nVi,m,nΔtviVi,m,n+1Vi,m,nΔx\displaystyle\frac{V_{i,m+1,n}-V_{i,m,n}}{\Delta t}-v_{i}\frac{V_{i,m,n+1}-V_{i,m,n}}{\Delta x} (4.15)
    μiVi,m,n+12Vi,m,n+Vi,m,n1Δx2+βiVi,m,nfi,m,n=0.\displaystyle-\mu_{i}\frac{V_{i,m,n+1}-2V_{i,m,n}+V_{i,m,n-1}}{{\Delta x}^{2}}+\beta_{i}V_{i,m,n}-f_{i,m,n}=0.
  • Step 2: Let the solution (Vi,m+1,n)1i3,1nNx(V_{i,m+1,n})_{1\leq i\leq 3,1\leq n\leq N_{x}} be given for the (m+1)(m+1)-th time step, we solve (4.15) for the solution at the mm-th time step and denote the result by (V^i,m,n)1i3,1nNx(\hat{V}_{i,m,n})_{1\leq i\leq 3,1\leq n\leq N_{x}}.

  • Step 3: We update again to account for the variational inequality arising from the switching control by the equation below,

    Vi,m,n=maxji(V^i,m,n,V^j,m,nhij).V_{i,m,n}=\max_{j\neq i}(\hat{V}_{i,m,n},\hat{V}_{j,m,n}-h_{ij}). (4.16)

By taking the procedures above, we can obtain the optimal payoff value V(t,x)V(t,x) for individuals starting at (t,x)(t,x), which estimates the largest expected value individual can obtain by executing the optimal control.

4.2 Switching region

In this subsection, we show how to represent the optimal strategy through the computation of value functions above. To achieve this goal, we recall the definitions of switching region and continuation region as below:

(Switching region)𝒮i={(t,x)[0,T]×[0,L]:Vi(t,x)=maxji(Vjhij)(t,x)}.\text{(Switching region)}\qquad\mathcal{S}_{i}=\{(t,x)\in[0,T]\times[0,L]:V_{i}(t,x)=\max_{j\neq i}(V_{j}-h_{ij})(t,x)\}.

The complement set of 𝒮i\mathcal{S}_{i} denoted by

(Continuation region)𝒞i={(t,x)[0,T]×[0,L]:Vi(t,x)>maxji(Vjhij)(t,x)}.\text{(Continuation region)}\qquad\mathcal{C}_{i}=\{(t,x)\in[0,T]\times[0,L]:V_{i}(t,x)>\max_{j\neq i}(V_{j}-h_{ij})(t,x)\}. (4.17)

We also denote the closed set representing the switching region from regime ii to regime jj by 𝒮ij\mathcal{S}_{ij}:

𝒮ij={(t,x)[0,T]×[0,L]:Vi(t,x)=(Vjhij)(t,x)for someji}.\mathcal{S}_{ij}=\{(t,x)\in[0,T]\times[0,L]:V_{i}(t,x)=(V_{j}-h_{ij})(t,x)\quad\text{for some}\quad j\neq i\}. (4.18)

With these notations, Proposition 3.3 shows that the diffusion process for an individual adopting the optimal switching strategy can be fully characterized by a controlled diffusion process where an individual switch between different diffusion processes according to its physical location (t,x)(t,x) and its current state ii, as given by (4.17) and (4.18). To numerically demonstrate this, we follow the following steps:

  • First, applying the numerical methods described in sub-section 4.1, we obtain the value functions Vi(t,x)V_{i}(t,x) for any fixed initial condition (t,x,i)(t,x,i).

  • Then, we compare Vi(t,x)V_{i}(t,x) for different regimes ii by taking switching cost hijh_{ij} (cost of switching from regime i to regime j) into consideration to draw the switching regions and continuing regions.

Refer to caption
Figure 3: (An illustration of switching regions) Consider three different realizations of the diffusion process (representing three distinct individuals). In path 1, the individual arrives at the green region, so it is optimal for the individual to switch to the waiting state i=3i=3 and stay there until it is time to switch to “direct flight” (which is the diffusion region i=2i=2). In the second path, the individual arrives at the red region at which point it is optimal to adopt direct flight immediately. (Note that once the individual adopts direct flight, no further switching is possible so it does not matter that it enters the green region. The above diagram only concerns individuals adopting switching modes i=1i=1 or i=3i=3.) Finally, in path 3, the individual does not enter the green or red region, so it is optimal for the individual to stay at diffusion mode of “detour flight” i=1i=1.

4.3 Implement Optimal Control

In this section, we discuss how to implement the optimal control α\alpha^{*} determined by the switching regions derived in the subsection 4.2 with stochastic simulations. Consider the stochastic processes that model the movement of individuals over time, governed by  (2.3). To analyze this process, we discretize (t,x)(t,x) domain into a lattice with step sizes Δtand Δx\Delta t\ \text{and }\Delta x, with absorbing boundaries (e.g. the process will be terminated once individual touch the boundary point x=Lx=L). Then individual movements will be modeled as a random walk with drift on the discretized domain with the transition probabilities p,prp_{\ell},p_{r} (and the probability 1=ppr1=p_{\ell}-p_{r} to stay put). We first fix spatial step size and then obtain the time step size according to spatial step to maintain numerical stability. After that, we derive for each regime ii the transition probabilities pp_{\ell}, prp_{r} for each regime. For simplicity, we present the formulas for i=1i=1, as the formulas for the other diffusion regime is similar.

ut=μ1uxx+v1ux,u_{t}=\mu_{1}u_{xx}+v_{1}u_{x},

let

Δx=h,Δt=δμ1+μ2h2,pl+pr2=μ1μ1+μ2,plpr=v1ΔtΔx=v1×Δx×Δt(Δx)2=v1Δxμ1+μ2.\begin{split}&\Delta x=h,\ \ \Delta t=\frac{\delta}{\mu_{1}+\mu_{2}}h^{2},\\ &\frac{p_{l}+p_{r}}{2}=\frac{\mu_{1}}{\mu_{1}+\mu_{2}},\\ &p_{l}-p_{r}=\frac{v_{1}\Delta t}{\Delta x}=v_{1}\times\Delta x\times\frac{\Delta t}{(\Delta x)^{2}}=\frac{v_{1}\Delta x}{\mu_{1}+\mu_{2}}.\end{split}

Then one gets

ut=pl+pr2×(Δx)2Δt×uxx+(plpr)×ΔxΔt×ux,pl=δ[μ1μ1+μ2v1h2(μ1+μ2)],pr=δ[μ1μ1+μ2+v1h2(μ1+μ2)].\begin{split}&u_{t}=\frac{p_{l}+p_{r}}{2}\times\frac{(\Delta x)^{2}}{\Delta t}\times u_{xx}+(p_{l}-p_{r})\times\frac{\Delta x}{\Delta t}\times u_{x},\\ &p_{l}=\delta\bigg{[}\frac{\mu_{1}}{\mu_{1}+\mu_{2}}-\frac{v_{1}h}{2(\mu_{1}+\mu_{2})}\bigg{]},\ \ p_{r}=\delta\bigg{[}\frac{\mu_{1}}{\mu_{1}+\mu_{2}}+\frac{v_{1}h}{2(\mu_{1}+\mu_{2})}\bigg{]}.\end{split}

Given the optimal control, individuals governed by the stochastic diffusion process (2.3) will start from (t,x,i)(t,x,i) and follow the random walk with transitional probabilities obtained above until one of the following cases happen:

  • Case 1: t=Tt=T or the individual hits the terminal site x=Lx=L.

  • Case 2: The first time t=τ1t=\tau_{1} when the individual enters (t,x){Sij}j(t,x)\in\{S_{ij}\}_{j} for some jij\neq i.

In the first case, the diffusion process will terminate. In the second case, the individual will continue the diffusion process for tτ1t\geq\tau_{1} with new regime state jj (and hence new values of (p,pr)(p_{\ell},p_{r}) for the approximate random walk), until one of the above scenarios happen again. Figure 3 is an illustrative example of stochastic processes described above. As the individual hits the switching region, switching time τn\tau_{n} and state after switching ιn\iota_{n} are recorded. When case 1 happens, a series of switching time and states (τn,ιn)n0(\tau_{n},\iota_{n})_{n\geqslant 0} can be obtained.

By repeating the above simulation multiple times, the quantitative statistics of bird migration can be obtained, which will be further analyzed in the following numerical investigation into specific biological problems.

Worth to mention that by our way of discretization, the spatitial step Δx\Delta x will be bounded by 2μv\frac{2\mu}{v}. For numerical stability, and the time step Δt\Delta t = δμ1+μ2(Δx)2\frac{\delta}{\mu_{1}+\mu_{2}}(\Delta x)^{2}, where δ\delta is a fixed small number, can be extremely small, which result in a large number of computations will be required for a single simulation. To resolve the problem, we combine nn time steps as one large step by first compute the probability distributions on the state space of steps moving forward [n,n+1,,0,1,,n1,n][-n,-n+1,\dots,0,1,\dots,n-1,n] steps and only check for update of regime for each individual every nn steps, with nn being a reasonable small integer so that the simulation still follows the optimal control strictly. With this new algorithms implemented, we were able to reduce the amount of computations needed by nn times.

4.4 Partial information

In reality, organisms make decisions based on perceived information. It is not possible in general, even for highly mobile organisms such as birds, to obtain perfect information of the environment, which depends on factors such as climate change, weather dynamics and so on. To model the partial information perceived by birds, we use a weighted average of historical payoffs, denoted by gg to reflect that information was based on the past migratory experiences of birds. Thus, we assume the bird possesses partial information gg as the average value of the perfect information GG. An extreme case is g=G(t)g=\fint G(t).

We propose a numerical scheme to model the optimal migration behavior under partial information as follows.

  • Step 1: Compute Vipartial(t,x)V^{partial}_{i}(t,x) as optimal control value with perceived terminal reward gg,

    Vipartial(t,x)=supα𝒜Et,x,i[tτT\displaystyle V^{partial}_{i}(t,x)=\sup_{\alpha\in\mathcal{A}}E^{t,x,i}\bigg{[}\int_{t}^{\tau\wedge T} eβi(s,x)(st)fi(s,Xst,x,i)dsτnτeβi(τn,x)(τnt)hln1,ln\displaystyle e^{-\beta_{i}(s,x)(s-t)}f_{i}(s,X_{s}^{t,x,i})\,ds-\sum_{\tau_{n}\leqslant\tau}e^{-\beta_{i}(\tau_{n},x)(\tau_{n}-t)}h_{l_{n-1},l_{n}} (4.19)
    +χ{τ<T}eβi(τ,x)(τt)g(τ)],\displaystyle+\chi_{\left\{\tau<T\right\}}e^{-\beta_{i}(\tau,x)(\tau-t)}g(\tau)\bigg{]},

    to obtain resulting optimal control (τ^n,ι^n)(\hat{\tau}_{n},\hat{\iota}_{n}) is expressed in terms of the switching region described in section 4.2. Note that the control is suboptimal with respect to the actual terminal payoff GG.

  • Step 2: Using the stochastic simulations introduced in subsection 4.3, where the actions of individual are governed by the optimal strategy obtained in Step 1 based on partial information gg, we can record a family of arriving time (τn)1nN(\tau^{n})_{1\leq n\leq N} at x=Lx=L. Then we use the Monte-Carlo method to compute the difference in expectations to corresponding to the perceived terminal payoff versus the actual terminal payoff function G(t)G(t).

    Et,x,i(g)1Nn=1Ng(τn),Et,x,i(G)1Nn=1NG(τn),E^{t,x,i}(g)\approx\frac{1}{N}\sum_{n=1}^{N}g{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}(\tau^{n})},\quad E^{t,x,i}(G)\approx\frac{1}{N}\sum_{n=1}^{N}G{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}(\tau^{n})}, (4.20)

    which implies

    Et,x,i(gG)=Et,x,i(g)Et,x,i(G).E^{t,x,i}(g-G)=E^{t,x,i}(g)-E^{t,x,i}(G). (4.21)
  • Step 3: Update the expected payoff value ViactualV^{actual}_{i}, accounting for the difference between gg (the bird’s perception of terminal reward) and GG (the actual terminal reward). To obtain the actual payoff

    Viactual(t,x)\displaystyle V^{actual}_{i}(t,x) =Et,x,i[tτTeβi(s,x)(st)fi(s,Xst,x,i)dsτnτeβi(τn,x)(τnt)hln1,ln\displaystyle=E^{t,x,i}\bigg{[}\int_{t}^{\tau\wedge T}e^{-\beta_{i}(s,x)(s-t)}f_{i}(s,X_{s}^{t,x,i})\,ds-\sum_{\tau_{n}\leqslant\tau}e^{-\beta_{i}(\tau_{n},x)(\tau_{n}-t)}h_{l_{n-1},l_{n}} (4.22)
    +χ{τ<T}eβi(τ,x)(τt)G(τ)]\displaystyle\qquad\qquad+\chi_{\left\{\tau<T\right\}}e^{-\beta_{i}(\tau,x)(\tau-t)}G(\tau)\bigg{]}
    =Vipartial(t,x)Et,x,i(gG)\displaystyle=V_{i}^{partial}(t,x)-E^{t,x,i}(g-G)

    under the control α\alpha in the form of switching regions obtained from Vipartial(t,x)V^{partial}_{i}(t,x).

From these process above, we can also observe how the switching behavior of the bird differ when the environment is different from their perceptions. That is, the difference of switching regions and optimal controls can be observed under the guidance of different environmental settings with various running rewards, terminal rewards, predation risks, etc.

5 Numerical Results

In this section, we address the questions posed in the introduction through numerical simulations. In this section, we use colormaps to illustrate the value of the value function V1(t,x)V_{1}(t,x) as well as the optimal control. Precisely, for Figures 6, 8, 8, 12(a) and 12(c), the red (resp. green) region represents the switching regions 𝒮12\mathcal{S}_{12} (resp. 𝒮13\mathcal{S}_{13}), corresponding to the switching from detour to direct flight (resp. detour to waiting at stopover site). The remaining region is 𝒞1\mathcal{C}_{1} (where the bird remains in the detour/slow-flight state). In the latter region, we use color map to indicate the dependence of the value function ViV_{i} on the time and space variables.

5.1 Deterioration of stopover site

Global warming frequently lead to the deterioration of stopover sites, rendering one or more of the stopover sites unsuitable for migrating birds. This subsection aims to study the impact of the deterioration of stopover sites in our model on the migration dynamics of shorebirds.

In accordance with the geographical set up laid out in Figures 1 and 2 in the introduction, we denote XkX_{k} as the first instance of xx value such that the kk-th stopover site is available. Then define Mk={x[0,L]:x[Xk,Xk+ϵk]}M_{k}=\{x\in[0,L]:x\in[X_{k},X_{k}+\epsilon_{k}]\}, as the locations of stopover sites in which the individual who are not in direct flight (recall that individual in direct flight has to continue in its journey until reaching x=Lx=L) can choose to switch to any other states. We denote the set 𝕄={Mkk=1,,n}.\mathbb{M}=\{M_{k}\mid k=1,\ldots,n\}. Specifically, we define

ϵk=L50,X1=L3ϵk2,X2=4L9ϵk2,X3=3L5ϵk2, and X3=5L6ϵk2.\epsilon_{k}=\frac{L}{50},~{}~{}X_{1}=\frac{L}{3}-\frac{\epsilon_{k}}{2},~{}~{}X_{2}=\frac{4L}{9}-\frac{\epsilon_{k}}{2},~{}~{}X_{3}=\frac{3L}{5}-\frac{\epsilon_{k}}{2},~{}~{}\text{ and }~{}~{}X_{3}=\frac{5L}{6}-\frac{\epsilon_{k}}{2}.

In addition, we define M0=[0,X1)M_{0}=[0,X_{1}) to be the positions of wintering site where individual can choose to switch between regime 1 and 3 (detour and waiting), but they can’t switch to regime 2 (direct flight) since it haven’t reached the sea. We fix the values of other parameters as in Table 1.

Table 1: Parameters
Parameter Baseline Value Description
LL 5301 kma Direct distance from Izembek to west coast of Mexico
v1v_{1} 373.33 km/dayb Detour flight velocity
v2v_{2} 560 km/dayc Direct flight velocity
v3v_{3} 0 Velocity of waiting
TT 72 daysd Total number of days covered
μ1\mu_{1} 150 km2/(day) Volatility associated with detour migration
μ2\mu_{2} 250 km2/(day) Volatility associated with direct migration
μ3\mu_{3} 0 Volatility associated with direct migration
β1\beta_{1} 0.00116/daye Mortality risk for detour migration
β2\beta_{2} 0.002/daye Mortality risk for direct migration
β3\beta_{3} 0.0002/daye Mortality risk at stop site 1 day
nn 3 Number of stopover sites available
f1f_{1} 0 Reward for regime 1: detour flight
f2f_{2} 0 Reward for regime 2: direct flight
f3f_{3} {0.00156dwintering site0.000907d1st stopover site0.00551d2nd stopover site0.00288d3rd stopover site0otherwise\displaystyle\begin{cases}0.00156{\textsuperscript{d}}&\text{wintering site}\\ 0.000907{\textsuperscript{d}}&\text{1st stopover site}\\ 0.00551{\textsuperscript{d}}&\text{2nd stopover site}\\ 0.00288{\textsuperscript{d}}&\text{3rd stopover site}\\ 0&\text{otherwise}\end{cases} Reward for regime 3: waiting
hijh_{ij} i,j(0,1,2,3)i,j\in(0,1,2,3) 0.06 Switch Cost
  • a

    According to [9], most Brent fly approximately 5301 km during winter migration. Here we consider spring migration and assume direct distance remains the same.

  • b

    We assume the detour velocity is lower than the direct velocity to account for the longer migration distance.

  • c

    According to [9], Brent migrate at a speed of about 80 kph, we assume on average birds migrate 7 hours per day, which translates to 560 km per day.

  • d

    Spring migration of black brants starts around mid-Jan and ends around early-April[20]. So spring migration lasts approximately 72 days.

  • e

    [38] estimates that the average annual survival rate for black brants is 0.84, so we approximate spring migration total mortality rate around 0.08, then averaged mortality rate is around 0.00116 per day.

  • f

    According to [24], the food abundance level is reported as follows: Baja California, Mexico (1,800 ha), Humboldt Bay (1,045 ha), Willapa Bay and Grays Harbor (6,650 ha), Boundary Bay (3,320 ha), and Izembek Lagoon, Alaska (16,000 ha). The reward is calculated by use the ratio of stopover site food level and Izembek food level divide by total migration duration.

Since availability of resources or breeding opportunity is sensitive to time, individuals that arrive too early (e.g., before eelgrass is exposed from winter ice) or too late (e.g., after eelgrass has been depleted) would compromise their ability to accumulate endogenous reserves [33]. Thus, we use a Gaussian function

G(t)=exp((tT2)22(T4)2)G(t)=\exp\left(-\frac{\left(t-\frac{T}{2}\right)^{2}}{2\left(\frac{T}{4}\right)^{2}}\right) (5.23)

to represent the terminal reward as a function of the arrival time at x=Lx=L in (4.14) which is maximized at the time t=T/2t=T/2. We set the number of stopover sites to be 3 to represent the opportunity to switch in the early, middle, or late stages of the migration process; see Figure 4.

Refer to caption
Figure 4: stopover sites

Then we impose different rewards for individuals to engage in staging at these three sites with the second staging site presenting the greatest reward, as shown in the Table 1. The running reward functions are denoted by fi,3(x,t)f_{i,3}(x,t) for i=1,2,3i=1,2,3, where the ‘33’ in the subscript means that the individual is in regime ‘33’ (i.e. the ’waiting’ regime). In the next section, we model the deterioration of the quality of site 2.

Modeling the effect of deteriorated stopover sites

The earlier snowmelt and onset of growing season in Arctic due to global warming [15, 34, 2] can result in earlier availability of food for black brants. Here, we only account for the impact of global warming to stopover site, and use λ[0,1]\lambda\in[0,1] to denote the level of deteriation. Precisely, the running reward function at the second stopover site is given as follows.

f2;3λ(x,t)=(1λ)f2;3(x,t),forλ[0,1].f^{\lambda}_{2;3}(x,t)=(1-\lambda)f_{2;3}(x,t),\\ \text{for}\ \lambda\in[0,1]. (5.24)

We choose to vary the quality of the second stopover site, as it is adopted by the optimal stopover sites for all individuals given by the optimal control as explained below. If there is no deterioration in stopover sites, we obtain the optimal controls as shown in Figure 6 by applying the methods introduced in subsection 4.1. Note that in Figure 6, although it is optimal to switch to staging at some (t,x)M3(t,x)\in M_{3}, individuals started at (0,0)(0,0) will always arrive at (t,x)M2(t,x)\in M_{2} first, at which the individual switches to direct flight and thus skipping over stopover site three. Hence, the third stopover site was never utilized if we only consider individuals started at (0,0)(0,0), even though the green region in Figure 6 indicates that it is optimal to stop there conditioned on an individual reaching it in detour flight state.

Next, we study the effect of stopover sites’ deterioration on (i) migratory payoff and optimal control; (ii) optimal stopover region.

5.1.1 Effect on migratory payoff and optimal control

In Figure 6, we plot the change in migratory payoff V(0,0)V(0,0) (conditioned on individual starting at (t,x)=(0,0)(t,x)=(0,0)) as the running benefit at the staging site deteriorated (i.e., λ\lambda increases from 0 to 11 in (5.24)).

Refer to caption
Figure 5: Optimal control at λ=0\lambda=0
Refer to caption
Figure 6: Migratory payoff against λ\lambda

It is observed in Figure 6 that V(0,0)V(0,0) decreases as λ\lambda increases in [0,0.5][0,0.5]. This implies a lower migratory payoff as deterioration worsens at a stopover site. We also observed that the slope approaches 0 after λ=0.5\lambda=0.5. Later on, we will see that this is due to the complete abandonment of the stopover site for λ\lambda large.

To explore the effects on optimal control, we observe the optimal controls at λ=0.5\lambda=0.5 (see Figure 8) and λ=0.55\lambda=0.55 (see Figure 8).

Refer to caption
Figure 7: Optimal control at λ=0.6\lambda=0.6
Refer to caption
Figure 8: Optimal control at λ=0.65\lambda=0.65

Compared to Figure 6 (λ=0\lambda=0, i.e, without deterioration), the switch region at stopover site 2 contracted significantly indicated by Figure 8. whereas the the region vanished completely in Figure 8, indicating that it is optimal for the migratory population to abandon stopover site 2. Since the stopover site is no longer utilized, further deterioration no longer impact the migratory payoff, which explains the change of slope in Figure 6. Note that there is a slight occurrence of switch region at switch site 0, which in our model is the point where bird reach the sea from wintering site. Thus, the occurrence of switch region at site 0 can be understand as a delay of departure.

5.1.2 Effect on optimal switching regions

To better investigate the behavior of individuals under the change of optimal control, we utilize stochastic simulation method introduced in sub-section 4.3. Here, we mainly observe the change in behavior at stopover site as the optimal control varies, which can be studied explicitly by observing the change of duration of staying at each stopover site Lk¯\overline{L_{k}}. By a stochastic simulation of migrations with total mm individuals with optimal control given by (4.17) and (4.18), Lk¯\overline{L_{k}} can be obtained by

Lk¯=j=1mDjkj=1m𝕀(Djk>0),\overline{L_{k}}=\frac{\sum_{j=1}^{m}\text{D}_{jk}}{\sum_{j=1}^{m}\mathbb{I}(D_{jk}>0)},\ \ (5.25)

where DjkD_{jk} denotes the duration of staying of the jj-th individual at the kk-th stopover site. Then, L¯jd\overline{L}^{d}_{j}, the average length of stay at a particular stopover site after incorporating deterioration, can be obtained by repeating the previous simulation procedures. Taking a different value of λ\lambda, the direct impact of a different level of deterioration on the length of stay at a stopover site can be studied as shown in Figure 9, in which the length of stay at each stopover site was displayed.

Refer to caption
Figure 9: Average length of stay at stopover site aginst λ\lambda

Based on our numerical results in Figure 9, we have three main observations as follows.

  • For the stopover site k=2k=2, beyond the critical deterioration level λ=0.5\lambda=0.5, the average duration of staying drops levels off at 0, which explains why V(0,0)V(0,0) is no longer senstive to further increase in deterioration level λ\lambda for λ>0.5\lambda>0.5 (see Figure 6). In [18], it is reported that migratory birds were forced to choose sub-optimal stopover sites due to the degradation of optimal stopover site, which result in less successful migrations.

  • When λ0.5\lambda\leq 0.5, there is no clear correlation between the length of stay at stopover site 2 and deterioration level λ\lambda, differing from the negative correlation observed in [33], where migratory individuals were observed to spend less time at the stopover site gradually.

  • As the deteriorated level λ\lambda increases, the duration of staying at stopover site 3 increases, which indicates the importance of it improved as site 2 was modeled to be deteriorated. stopover site 0 and 1 was never be utilized, which is due to the running rewards at stopover site 0 and 1 are much smaller compared to other stopover sites and the distance between them and other sites are small relative to migratory velocities, so there is no advantage staying at stopover site 0 and 1.

Next, we introduced the modified terminal reward

Gλ(t,γ)=exp((t(T2+γTλ))22(T4)2)G^{\lambda}(t,\gamma)=\exp\left(-\frac{\left(t-\left(\frac{T}{2}+\gamma\cdot T\cdot\lambda\right)\right)^{2}}{2\left(\frac{T}{4}\right)^{2}}\right) (5.26)

which shift the peak timing of terminal reward G in 4.14 to left as λ\lambda increase, and γ\gamma is the shift magnitude such that Gλ(t)G^{\lambda}(t) take its maximum value at T2γT(1λ)\frac{T}{2}-\gamma T(1-\lambda), where γ[0,12]\gamma\in[0,\frac{1}{2}]. In the following analysis, γ=110\gamma=\frac{1}{10} was chosen so that the peak of G(t)G(t) was shift to the left by T10\frac{T}{10}, which indicates onset of growing season is 7.2 days earlier in the model. This choice is motivated by [41], which estimated that the earliest onset was advancing at 7.64 days/decade. Here λ\lambda was used again since deterioration level and earlier onset of growing season in Arctic can both be attributed to global warming. Following the same procedures above, we obtain the optimal control and duration of staying at each stopover site.

Refer to caption
Figure 10: Migratory payoff after adjusting terminal reward
Refer to caption
Figure 11: Average duration of stay at stopover site against λ\lambda after effect on terminal

Compared with Figure 6 and 9, one observation is that the migratory payoff continue to decrease even after stopover site 2 was abandoned and birds choose to wait at stopover site 3. This is due to that as the onset of growing season in Arctic become earlier and peak terminal reward shift left, there is less time available for birds to rest at any stopover sites, which results in less reward from resting at any stopover site and thus decrease migratory payoff even when the deteriorated stopover site was no longer utilized. Another observation is that the duration of staying at stopover sites 2 and 3 both decreased as the deterioration level increased before the critical deterioration level. This is consistent with the observation by [33] and [15] that birds tended to stay for a shorter period of time at stopover sites due to effect of global warming.

5.2 Perfect vs Imperfect information

In previous sections, we have observed how individuals maximize their payoff by choosing a migration strategy, which is based on the assumption that the individual agent has perfect information of the environment and can make prediction about their journey. However, as discussed in the introduction, individual birds typically do not have access to perfect environmental information. While they can rely on past experience to estimate the conditions in their migratory route based on the perception, the actual environmental condition is likely subject to additional factors such as stochasticity/noise, or other changes due to human activities and global climate change. In any case, individual birds must make decisions based on imperfect information, leading to deviations between the expected and actual rewards.

In some circumstances, the individual bird can gain access to more accurate environmental condition at the destination by utilizing a stopover site that is closer to the destination, in contrast to making a long distance direct flight. For example, the Icelandic whimbrel (Numenius phaeopus islandicus) migrates from Iceland to West Africa during spring migration, utilizing West Europe as a stopover site [13]. Carneiro et al. [7] identified two primary spring migration strategies employed by this species: a direct flight from wintering to breeding sites, versus the incorporation of a stopover site. The latter strategy is enables the birds to evaluate terminal site weather conditions at the stopover, thereby reducing weather-related risks, and is generally preferred.

Here, we attempt to address the following biological question:

  • How does weather dynamics at the terminal site impact bird migration when more accurate weather information can be gained when utilizing a stopover site?

For clarity, denote by g(t)g(t) the perceived terminal reward function (based on imperfect information) and denote by G(t)G(t) the actual terminal reward function (based on perfect information). To address the aforementioned question, we investigate the effect of information on both the expected payoff and migration strategy, which are analyzed in two aspects corresponding to the following subsections:

In Subsection 5.2.1, we explore the effect of noise in the terminal reward function. Precisely, we assume that the actual reward function is a perturbation of the perceived reward:

G(t,x)=g(t,x)+Aκ(t),G(t,x)=g(t,x)+A\kappa(t), (5.27)

where κ(t)\kappa(t) is an oscillatory function with high frequency and small amplitude. The parameter AA governs the intensity variation of the fluctuation with values ranging from 0 to 1.

In Subsection 5.2.2, we are motivated by [7] to explore the situation where individuals have access to the perfect information G(t)G(t) by using stopover site(s). Here, we mainly discuss about two scenarios as follows:

  • Scenario1: The perceived terminal reward function g(t)g(t) is a coarse-grained version of the actual reward funtion G(t)G(t), i.e. represented as a projection of G(t)G(t) into the space of step functions:

    gn(t)=k=0nχ{tkttk+1}tktk+1G𝑑t,where 0=t0tk=kTntn=T.g_{n}(t)=\sum^{n}_{k=0}\chi_{\left\{t_{k}\leq t\leq t_{k+1}\right\}}\fint_{t_{k}}^{t_{k+1}}G\,dt,\quad\text{where }0=t_{0}\leq\dots\leq t_{k}=\frac{kT}{n}\leq\dots\leq t_{n}=T. (5.28)
  • Scenario2: Climate change can cause the actual weather to deviate significantly from the individual’s experience or perception of the past. For instance, global warming may shift the timing and intensity of the green-up peak at the terminal site [31]. The relationship of g(t)g(t) and G(t)G(t) can be written as follows.

    G(t)=g(t+tmove),G(t)=g(t+t_{move}), (5.29)

    where tmovet_{move} characterizes how much the “green up” time arrives earlier.

We apply the numerical methods introduced in Section 4 as follows:

  • First, we examine the difference between expected and real optimal value at (0,0)(0,0) guided by optimal control under partial information. We will also see the change of the difference with strength AA increasing. See details in Subsection 5.2.1;

  • Second, we observe how the optimal control (i.e. the collection of switching regions) changes if the agent can access (and optimize with) the perfect information G(t)G(t) after spending time at the stopover site. We will discuss two specific cases as mentioned above. See details in Subsection 5.2.2.

The parameters used in the following simulations are summarized in Table 2.

Table 2: Parameters
Parameter Baseline Value Description
LL 6450 kma Distance from Iceland to Guinea in West Africa
v1v_{1} 277.56 km/dayb Detour flight velocity
v2v_{2} 293.328 km/dayb Direct flight velocity
v3v_{3} 0 Velocity of waiting
TT 70 daysa Total number of days covered
μ1\mu_{1} 145 km2/(day) Volatility associated with detour migration
μ2\mu_{2} 150 km2/(day) Volatility associated with direct migration
μ3\mu_{3} 0 Volatility associated with direct migration
β1\beta_{1} 0.0005 /day Mortality risk for detour migration
β2\beta_{2} 0.00055 /day Mortality risk for direct migration
β3\beta_{3} 0.0004 /day Mortality risk for waiting at the stopover site
nn 1c Number of stopover sites available (excluding wintering site)
f1f_{1} 0 Reward for regime 1: detour flight
f2f_{2} 0 Reward for regime 2: direct flight
f3f_{3} {0.00165dwintering site0.00276dstopover site0otherwise\displaystyle\begin{cases}0.00165{\textsuperscript{d}}&\text{wintering site}\\ 0.00276{\textsuperscript{d}}&\text{stopover site}\\ 0&\text{otherwise}\end{cases} Reward for regime 3: waiting
hijh_{ij} (i,j{1,2,3,4}i,j\in\{1,2,3,4\}) {0.001ewintering site0.05estopover site\displaystyle\begin{cases}0.001{\textsuperscript{e}}&\text{wintering site}\\ 0.05{\textsuperscript{e}}&\text{stopover site}\end{cases} Switching cost
  • a

    We consider the spring migration of Numenius phaeopus islandicus from West Africa to Iceland. According to [6], the migration distance is around 6450 km with a perturbation of 118 km over a span of 70 days.

  • b

    According to [8] and [6], the reasonable minimum and maximum ground speed of Numenius phaeopus islandicus in spring are 277.56 km/day and 293.328 km/day. As we fix the distance for both detour and direct migration, we need to distinguish the two regimes by velocity. To this end, we take the biggest reasonable variance, assigning 277.56 km/day for detour and 293.328 km/day for direct migration.

  • c

    Although according to [6], Numenius phaeopus islandicus do have several stopover sites to choose during the spring migration, like Ireland, western Britain, northwest France, and Portugal, the phenomena observed in [7] only involves one stopover site. Here, we put it at the position around 23L\frac{2}{3}L , representing the western Europe as a whole.

  • d

    The reward is scaled consistently with Section 5.1.

  • e

    Switching costs are uniformly set to approximately 10%10\% of the terminal reward, except outside the wintering and stopover sites, where they are set to infinity. Direct flight is only allowed at the wintering site. After introducing regime 44 in Subsection 5.2.2, h34h_{34} is the only finite switching cost at the stopover site.

5.2.1 Diffusion guided by partial information

In this subsection, we explore the impact of the noise in the terminal reward on the optimal value. To this end, we impose (5.27) with

g(t)=exp((t34T)22(T8)2)andκ(t)=0.1sin(140πTt).g(t)=\exp\bigg{(}-\frac{(t-\frac{3}{4}T)^{2}}{2(\frac{T}{8})^{2}}\bigg{)}\quad\text{and}\quad\kappa(t)=0.1\sin\bigg{(}\frac{140\pi}{T}t\bigg{)}.

We will compute two value functions. One is the value function VimperfectV^{imperfect} which the individual agent uses to optimize its strategy. This can be computed using the Hamilton-Jacobi-Bellman equations, and can be interpreted as the perceived reward by the individual. The other value function is the actual value function VperfectV^{perfect} that takes into account that the actual terminal reward G(t)G(t) is different from the perceived reward g(t)g(t) that the individual agent used in the optimization process. Hence VperfectVimperfectV^{perfect}-V^{imperfect} represents the effect of information in and the mismatch between the perceived expected payoff and the actual expected payoff. We will study the relationship between the amplitude of the fluctuation AA and the difference of value functions D:=VperfectVimperfectD:=V^{perfect}-V^{imperfect} at (0,0)(0,0) (conditioned on the individual starting at (t,x)=(0,0)(t,x)=(0,0)).

First, we perform the optimization of the individual agent based on the perceived terminal reward function g(t)g(t), i.e. following the steps outlined in Subsection 4.4 to compute the optimal control αimperfect\alpha_{imperfect} by solving for the solutions Vi(t,x)V_{i}(t,x) of the PDE systems (4.14) with G(t)G(t) replaced by g(t)g(t), and identifying the corresponding switching regions SijS_{ij}. Note that it is enough show the switching regions conditioned on the individual being states 1 (detour flight) or 3 (waiting) since the individual stays in state 2 (direct flight) once it adopts direct flight. The switching region from states 11 to {2,3}\{2,3\} and from 33 to {1,2}\{1,2\} are shown respectively in Figure 12(a) and Figure 12(b). Hereafter we call these switching regions (which is calculated based on imperfect information) αimperfect\alpha_{imperfect}. It is demonstrated that most birds prefer to choose detour flight at first. After arriving at the stopover site, they tend to switch to waiting state and wait for the optimal time (approximately t=40t=40) to continue their flight and finish their migration.

Refer to caption
(a) Regime 1: detour flight
Refer to caption
(b) Regime 3: waiting
Refer to caption
(c) Stochastic simulation
Refer to caption
(d) Distribution of arrival time
Figure 12: Result under imperfect information

Next, we perform Monte Carlo simulation of the stochastic diffusion process with the control αimperfect\alpha_{imperfect}, repeated N=500N=500 times, to produce a family of possible migrating routes (see in Figure 12(c)). From these simulations, we obtain the statistics of arrival time at x=Lx=L based on control αimperfect\alpha_{imperfect}, as shown in the violinplot in Figure 12(d). It is observed that this distribution qualitatively aligns with the shape of the perceived terminal green-up timing g(t)g(t). As depicted in the figure, most birds choose to arrive at terminal site around the peak of green-up time to accept better terminal reward. This shows that individual’s experience and perception, embedded in partial information, determine the individuals’ decision-making to a large extent.

Then, we estimate the difference of optimal value at (0,0)(0,0) under perfect and imperfect information. We first use the following equation to calculate Vimperfect(0,0)V^{imperfect}(0,0):

Vimperfect(0,0)=Eαimperfect[0τTeβi(s,0)sfi(s,Xs0,0,i)𝑑sτnτeβi(τn,0)τnhln1,ln+χ{τ<T}eβi(τ,0)τg(τ)].\begin{split}&V^{imperfect}(0,0)\\ &\ \ =E^{\alpha_{imperfect}}\bigg{[}\int_{0}^{\tau\wedge T}e^{-\beta_{i}(s,0)s}f_{i}(s,X_{s}^{0,0,i})\,ds-\sum_{\tau_{n}\leqslant\tau}e^{-\beta_{i}(\tau_{n},0)\tau_{n}}h_{l_{n-1},l_{n}}+\chi_{\left\{\tau<T\right\}}e^{-\beta_{i}(\tau,0)\tau}g(\tau)\bigg{]}.\end{split} (5.30)

However, in reality, bird is led by partial information but gets actual reward as payoff, which is given by the following equation:

Vperfect(0,0)=Eαimperfect[0τTeβi(s,0)sfi(s,Xs0,0,i)𝑑sτnτeβi(τn,0)τnhln1,ln+χ{τ<T}eβi(τ,0)τG(τ)].\begin{split}&V^{perfect}(0,0)\\ &\ \ =E^{\alpha_{imperfect}}\bigg{[}\int_{0}^{\tau\wedge T}e^{-\beta_{i}(s,0)s}f_{i}(s,X_{s}^{0,0,i})\,ds-\sum_{\tau_{n}\leqslant\tau}e^{-\beta_{i}(\tau_{n},0)\tau_{n}}h_{l_{n-1},l_{n}}+\chi_{\left\{\tau<T\right\}}e^{-\beta_{i}(\tau,0)\tau}G(\tau)\bigg{]}.\end{split} (5.31)

Therefore, with the distribution obtained above, we can compute the expectation of difference value of value function at (0,0)(0,0), denoted as DD, with equation (5.30) and (5.31).

D:=Vimperfect(0,0)Vperfect(0,0)=1Nk=1Neβi(τk,0)τk(g(τk)G(τk)).D:=V^{imperfect}(0,0)-V^{perfect}(0,0)=\frac{1}{N}\sum^{N}_{k=1}e^{-\beta_{i}(\tau_{k},0)\tau_{k}}(g(\tau_{k})-G(\tau_{k})). (5.32)

Then we proceed to test the influence of the amplitude AA on the variance of the difference over all the stochastic simulations. Denote the variance as VarVar, whose calculation formula is

Var:=1Nk=1N(eβi(τk,0)τk(g(τk)G(τk))D)2.Var:=\frac{1}{N}\sum^{N}_{k=1}(e^{-\beta_{i}(\tau_{k},0)\tau_{k}}(g(\tau_{k})-G(\tau_{k}))-D)^{2}. (5.33)

We observed that with the amplitude AA increasing, differences of optimal value will exhibit more variability. The result is consistent with our speculation, as shown in Figure 14. From a biological point, this shows that with more intensive noise mixed in the terminal weather conditions, the difference of birds’ expectation and reality will show extreme variability, thus increasing the volatility of getting different payoff with birds’ expectation. Therefore, under this circumstance, a stopover site for obtaining perfect information becomes necessary to reduce the risk of mismatch.

Refer to caption
Figure 13: Variance value VarVar under different amplitude AA
Refer to caption
Figure 14: Mean value D under different partition number n

5.2.2 When individual can gain information after using stopover site

In this subsection, we propose a new method to explore the effect of information on the optimal switching strategy changes. Specifically, we will model an individual as entering a new state with perfect information (regarding the quality of terminal site) if it chooses to rest at a particular stopover site.

The numerical method for this case is outlined as follows. To address the state of perceiving perfect information after waiting at the stopover site, we introduce a new regime 4, marked in color magenta in graph, to the PDE system with its terminal reward replaced by perfect information G(t)G(t). The parameters v4,μ4,β4v_{4},\mu_{4},\beta_{4} of regime 4 are the same with regime 1 (i.e., detour flight). The individual who gained information cannot lose information anymore, i.e. for switching costs, we only set h34=0.05h_{34}=0.05 at the stopover site and set any other switching costs related to regime 44 at any other places to \infty. In this updated system, we use the same numerical approach described in Section 4 to compute the optimal control. Subsequently, we perform stochastic simulations. Initially, the bird follows the control derived under imperfect information, which corresponds to regimes 1, 2, and 3. If the bird chooses to wait at the stopover site, it switches to Regime 4 and follows the control derived under perfect information thereafter. This framework effectively implements the process of information-gathering at the stopover site.

We recall that g(t)g(t) is the projection of G(t)G(t) into step functions space in Scenario1. In Scenario2, G(t)G(t) has the same shape as g(t)g(t) but with the peak arriving earlier, as defined in (5.28) and (5.29).

Scenario1: Projection of G(t)G(t) onto step functions spaces

We first focus on Scenario1. Fix the interval parameter n=2n=2 and choose

G(t)={4T(t(34TT4)),if x[12T,34T],4T(t(34T+T4)),if x[34T,T],0,otherwise,G(t)=\begin{cases}\frac{4}{T}(t-(\frac{3}{4}T-\frac{T}{4})),&\text{if }x\in[\frac{1}{2}T,\frac{3}{4}T],\\ -\frac{4}{T}(t-(\frac{3}{4}T+\frac{T}{4})),&\text{if }x\in[\frac{3}{4}T,T],\\ 0,&\text{otherwise},\end{cases}

which is a single-peak function. Then when n=2n=2, function g(t)=12g(t)=\frac{1}{2} is a constant within the interval [12T,T][\frac{1}{2}T,T] and 0 elsewhere. For the new PDE system, we set partial information g(t)g(t) as the terminal reward for regime 1,2,3 and perfect information G(t)G(t) as the terminal reward for regime 4 to simulate the process of perceiving perfect information at the stopover site. By solving the solution to the HJB system, we can get the optimal control αperfect\alpha_{perfect}, under which we can perform further stochastic simulations.

Refer to caption
(a) Regime 1: detour flight
Refer to caption
(b) Regime 3: waiting
Refer to caption
(c) Stochastic simulation
Refer to caption
(d) Arrival time distribution
Figure 15: Result incorporating perfect information (Scenario 1)

The switching region corresponding to the control αperfect\alpha_{perfect} is shown in Figure 15(a) and Figure 15(b). Combined with the simulation results in Figure 15(c), it is demonstrated that birds prefer to choose to wait at the stopover site to gain an access to perfect information as it in an increased overall payoff.

The statistics of arrival time at x=Lx=L is recorded as shown in the violinplot in Figure 15(d). The shape of the distribution of arrival time aligns exactly with the shape of perfect information G(t)G(t), or otherwise, birds may choose to arrive as soon as possible due to the constant reward at the terminal site, causing a mismatch with the actual peak of green-up time of terminal reward. This shows perceiving perfect information at the stopover site improves the individual’s expected payoff by decreasing bird’s risk of mismatching the green-up timing at the terminal site.

Next, we proceed to see how it will influence the expected difference of optimal value with the increase of the partition number nn. We observe that with the increase of partition number nn, the difference of optimal value will gradually be close to 0, as shown in Figure 14. This result aligns with our expectation because as nn increases, the partial information g(t)g(t) gradually converges to perfect information G(t)G(t) under certain norm. Consequently, this convergence naturally leads to a smaller discrepancy in the expected optimal value.

In summary, in Scenario1, i.e.

gn(t)=k=0nχ{tkttk+1}tktk+1G𝑑t,g_{n}(t)=\sum^{n}_{k=0}\chi_{\left\{t_{k}\leq t\leq t_{k+1}\right\}}\fint_{t_{k}}^{t_{k+1}}G\,dt,

individual bird prefers to switch to waiting at the stopover site to avoid the risk of mismatch. The improvement of expected payoff is greatest when n=0n=0, i.e. the perceived terminal reward satisfies g(t)=G𝑑tg(t)=\fint G\,dt. However, as Additionally, the expectation of difference of optimal value gradually approaches 0 as the nn\to\infty, so that the perceived terminal reward approaches the actual reward gn(t)G(t)g_{n}(t)\to G(t) almost everywhere.

Scenario2: Influence of global warming

We take

g(t)=exp((t34T)22(T8)2)andG(t)=exp((t12T)22(T8)2),g(t)=\exp\bigg{(}-\frac{(t-\frac{3}{4}T)^{2}}{2(\frac{T}{8})^{2}}\bigg{)}\ \ \text{and}\ \ G(t)=\exp\bigg{(}-\frac{(t-\frac{1}{2}T)^{2}}{2(\frac{T}{8})^{2}}\bigg{)},

where the peak green-up timing is shifted earlier by tmove=T4t_{move}=\frac{T}{4} due to the global warming effect.

Refer to caption
(a) Regime 1: detour flight
Refer to caption
(b) Regime 3: waiting
Refer to caption
(c) Stochastic simulation
Refer to caption
(d) Comparison of arrival time
Figure 16: Result incorporating perfect information (Scenario2)

With the same PDE systems described in Scenario1, we can compute the optimal control αperfect\alpha_{perfect} and its corresponding switching region, as shown in Figure 16(a) and Figure 16(b). Then we move on to simulate the diffusion process, with the result shown in Figure 16(c).

We observe that birds changes their behavior after they obtain more accurate information of the terminal reward function. Indeed, the optimal switching control αimperfect\alpha_{imperfect} aims at allowing individual to arrive at x=Lx=L around the perceived peak green-up timing which is t=34Tt=\tfrac{3}{4}T. This is illustrated in Figure 12. In contrast, individuals which gained access to better information at the stopover site deviates the optimal control αimperfect\alpha_{imperfect} by setting off earlier and aims at reaching the terminal site at t=12Tt=\tfrac{1}{2}T.

To show the difference of arrival time more clearly, we record the arrival time for the individuals shown in Figure 16(c) in the statistical way as before. The statistics of their arrival time is shown in Figure 16(d), which perfectly match with our optimal control shown in Figure 16(a) and 16(b), showing the influence of perceiving perfect information at the stopover site. With the violinplot shown in Figure 16(d), we can observe that birds waiting at the stopover site changes their strategies and try to reach the terminal site at around 12T\frac{1}{2}T, aligning with the peak time of actual terminal reward. By contrast, as shown in Figure 12(d), birds having waited at the stopover site mostly arrived at 34T\frac{3}{4}T, showing they are guided by partial information and thus would end with an intensive mismatch with the actual terminal reward.

To summarize our result for Scenario 2, it is demonstrated that as a plausible response to the increased mismatch between perceived and actual environmental information (due to, e.g. increased stochasticity of the global climate and/or global warming), individual may utilize certain key stopover site to gain a better estimation of the weather condition at the terminal site, which can in turn help them decrease the mismatch in the timing of arrival with the peak green-up time at the terminal site.

In conclusion, under global climate change, it is not enough for bird population to rely solely on past experience or perception in their selection of migratory strategies. In fact, stopover sites play an increasingly important role for birds in interpreting the weather conditions along their migratory route, helping them to better select migratory strategies. Therefore, it is even more important to protect and preserve these key stopover locations to ensure the survival of migratory bird populations.

References

  • [1] T. Alerstam, Optimal bird migration revisited, Journal of Ornithology, 152 (2011), pp. 5–23.
  • [2] U. S. Bhatt, D. A. Walker, M. K. Raynolds, J. C. Comiso, H. E. Epstein, G. Jia, R. Gens, J. E. Pinzon, C. J. Tucker, C. E. Tweedie, et al., Circumpolar arctic tundra vegetation change is linked to sea ice decline, Earth Interactions, 14 (2010), pp. 1–20.
  • [3] D. Bickford, S. D. Howard, D. J. Ng, and J. A. Sheridan, Impacts of climate change on the amphibians and reptiles of southeast asia, Biodiversity and conservation, 19 (2010), pp. 1043–1062.
  • [4] B. Boufoussi, S. Hamadène, and M. Jakani, Viscosity solutions of system of pdes with interconnected obstacles and nonlinear neumann boundary conditions, Journal of Mathematical Analysis and Applications, 522 (2023), p. 126947.
  • [5] C. J. Butler, The disproportionate effect of global warming on the arrival dates of short-distance migratory birds in north america, Ibis, 145 (2003), pp. 484–495.
  • [6] C. Carneiro, T. G. Gunnarsson, and J. A. Alves, Faster migration in autumn than in spring: seasonal migration patterns and non-breeding distribution of icelandic whimbrels numenius phaeopus islandicus, Journal of Avian Biology, 50 (2019).
  • [7] C. Carneiro, T. G. Gunnarsson, and J. A. Alves, Linking weather and phenology to stopover dynamics of a long-distance migrant, Frontiers in Ecology and Evolution, 8 (2020), p. 145.
  • [8] G. Castro and J. Myers, Flight range estimates for shorebirds, The Auk, 106 (1989), pp. 474–476.
  • [9] C. P. Dau, The fall migration of pacific flyway brent branta bernicla in relation to climatic conditions, Wildfowl, 43 (1992), pp. 80–95.
  • [10] B. Djehiche, S. Hamadène, and A. Popier, A finite horizon optimal multiple switching problem, SIAM J. Control Optim., 48 (2009), pp. 2751–2770.
  • [11] W. Fleming and H. Soner, Controlled Markov Processes and Viscosity Solutions, Springer New York, NY, 2006.
  • [12] S. Gourley and R. Liu, An age-structured model of bird migration, Mathematical Modelling of Natural Phenomena, 10 (2015), pp. 61–76.
  • [13] T. G. Gunnarsson and G. A. Gumundsson, Migration and non-breeding distribution of icelandic whimbrels numenius phaeopus islandicus as revealed by ringing recoveries, Wader Study, 123 (2016), pp. 44–48.
  • [14] A. I. Houston and J. M. McNamara, Models of adaptive behaviour: an approach based on state, Cambridge University Press, 1999.
  • [15] J. W. Hupp, D. H. Ward, D. X. Soto, and K. A. Hobson, Spring temperature, migration chronology, and nutrient allocation to eggs in three species of arctic-nesting geese: Implications for resilience to climate warming, Global Change Biology, 24 (2018), pp. 5056–5071.
  • [16] L. Jenni and M. Kéry, Timing of autumn bird migration under climate change: Advances in long-distance migrants, delays in short-distance migrants, Proceedings: Biological Sciences, 270 (2003), pp. 1467–1471.
  • [17] I. Kharroubi, Optimal switching in finite horizon under state constraints, SIAM J. Control Optim., 54 (2016), pp. 2202–2233.
  • [18] P. Lehikoinen, A. Lehikoinen, M. Mikkola-Roos, and K. Jaatinen, Counteracting wetland overgrowth increases breeding and staging bird abundances, Scientific Reports, 7 (2017), p. 41391.
  • [19] S. Lenhart and J. T. Workman, Optimal control applied to biological models, Chapman and Hall/CRC, 2007.
  • [20] T. L. Lewis, D. H. Ward, and J. S. e. a. Sedinger, Brant (branta bernicla), version 1.0, 2020. Accessed on 11/27/2024.
  • [21] M. Ludkovski, Optimal switching with applications to energy tolling agreements, 2005.
  • [22] N. Lundström and M. Olofsson, Systems of fully nonlinear parabolic obstacle problems with neumann boundary conditions, Applied Mathematics & Optimization, 86 (2022).
  • [23] M. Mangel, Stochastic dynamic programming illuminates the link between environment, physiology, and evolution, Bulletin of Mathematical Biology, 77 (2015), pp. 857–877.
  • [24] J. E. Moore, M. A. Colwell, R. L. Mathis, and J. M. Black, Staging of Pacific flyway brant in relation to eelgrass abundance and site isolation, with special consideration of humboldt bay, California, Biological Conservation, 115 (2004), pp. 475–486.
  • [25] C. Nilsson, R. H. Klaassen, and T. Alerstam, Differences in speed and duration of bird migration between spring and autumn, The American Naturalist, 181 (2013), pp. 837–845.
  • [26] C. Parzani, and S. Puechmorel, On a Hamilton‐Jacobi‐Bellman approach for coordinated optimal aircraft trajectories planning. Optim.Contr.Appl.Method, 39 (2016), pp.  933–948.
  • [27] P. E. Protter, Stochastic Integration and Differential Equations, Springer Berlin Heidelberg, 2005.
  • [28] H. Pham, On the smooth-fit property for one-dimensional optimal switching problem, In Séminaire de Probabilités XL. Lecture Notes in Mathematics. 1899 (2007), pp. 187–201.
  • [29] H. Pham, Continuous-time Stochastic Control and Optimization with Financial Applications, Springer Berlin, Heidelberg, 2009.
  • [30] J. Purcell and A. Brodin, Factors influencing route choice by avian migrants: a dynamic programming model of Pacific brant migration, Journal of Theoretical Biology, 249 (2007), pp. 804–816.
  • [31] E. P. Robertson, F. A. La Sorte, J. D. Mays, P. J. Taillie, O. J. Robinson, R. J. Ansley, T. J. O’Connell, C. A. Davis, and S. R. Loss, Decoupling of bird migration from the changing phenology of spring green-up, Proceedings of the National Academy of Sciences, 121 (2024), p. e2308433121.
  • [32] V. Salewski and B. Bruderer, The evolution of bird migration—a synthesis, Naturwissenschaften, 94 (2007), pp. 268–279.
  • [33] B. D. Smith, K. R. Hagmeier, W. S. Boyd, N. K. Dawe, T. D. Martin, and G. L. Monty, Trends in volume migration chronology in spring staging Pacific black brant, The Journal of Wildlife Management, 76 (2012), pp. 593–599.
  • [34] R. S. Stone, E. G. Dutton, J. M. Harris, and D. Longenecker, Earlier spring snowmelt in northern alaska as an indicator of climate change, Journal of Geophysical Research: Atmospheres, 107 (2002), pp. ACL–10.
  • [35] Y. Vardanis, J.-Å. Nilsson, R. H. Klaassen, R. Strandberg, and T. Alerstam, Consistency in long-distance bird migration: contrasting patterns in time and space for two raptors, Animal Behaviour, 113 (2016), pp. 177–187.
  • [36] G.-R. Walther, E. Post, P. Convey, A. Menzel, C. Parmesan, T. J. Beebee, J.-M. Fromentin, O. Hoegh-Guldberg, and F. Bairlein, Ecological responses to recent climate change, Nature, 416 (2002), pp. 389–395.
  • [37] D. H. Ward, C. P. Dau, T. L. Tibbitts, J. S. Sedinger, B. A. Anderson, and J. E. Hines, Change in abundance of Pacific brant wintering in Alaska: evidence of a climate warming effect?, Arctic, (2009), pp. 301–311.
  • [38] D. H. Ward, E. A. Rexstad, J. S. Sedinger, M. S. Lindberg, and N. K. Dawe, Seasonal and annual survival of adult Pacific brant, The Journal of Wildlife Management, (1997), pp. 773–781.
  • [39] R. C. Ydenberg, R. W. Butler, and D. B. Lank, Effects of predator landscapes on the evolutionary ecology of routing, timing and molt by long-distance migrants, Journal of Avian biology, 38 (2007), pp. 523–529.
  • [40] H. Yoshioka, T. Tanaka, F. Aranishi, T. Izumi, and M. Fujihara, Stochastic optimal switching model for migrating population dynamics, Journal of Biological Dynamics, 13 (2019), pp. 706–732.
  • [41] J. Zheng, G. Jia, and X. Xu, Earlier snowmelt predominates advanced spring vegetation greenup in alaska, Agricultural and Forest Meteorology, 315 (2022), p. 108828.
  • [42] S.P. Zhu, and G.Y. Ma., An analytical solution for the HJB equation arising from the Merton problem, Int. J. Financ. Eng., (2018), pp. 1850008.