2.1. The underlying dynamics

Let $(\Omega, \mathcal{F}, \{\mathcal{F}_t\}_{t \geq 0}, \mathbb{P})$ be a filtered probability space which satisfies the usual conditions. We consider an uncontrolled process X defined on $(\Omega, \mathcal{F}, \{\mathcal{F}_t\}_{t \geq 0}, \mathbb{P})$ that evolves in $\mathbb{R}_+$ and is modelled as a solution to regular linear Itô diffusion

\begin{equation*}{\rm d} X_t=\mu(X_t){\rm d} t+ \sigma(X_t){\rm d} W_t, \quad \quad X_0 = x,\end{equation*}

where $W_t$ is the Wiener process and the functions $\mu,\sigma\,{:}\, (0,\infty) \to \mathbb{R}$ are continuous and satisfy the condition $\int_{x-\varepsilon}^{x+\varepsilon} \frac{1+|\mu(y)|}{\sigma^2(y)}{\rm d} y < \infty$. These assumptions guarantee that the diffusion has a unique weak solution (see [Reference Karatzas and Shreve22, Section 5.5]). Although we consider the case where the process evolves in $\mathbb{R}_+$, we remark that this is done only for notational convenience, and the results would remain the same, with obvious changes, if the state space were replaced with any interval of $\mathbb{R}$.

We define the second-order linear differential operator $\mathcal{A}$ that represents the infinitesimal generator of the diffusion X as

\begin{equation*}\mathcal{A} = \mu(x) \frac{{\rm d}}{{\rm d} x} + \frac{1}{2} \sigma^2(x)\frac{{\rm d}^2}{{\rm d} x^2},\end{equation*}

and for a given $\lambda > 0$ we respectively denote the increasing and decreasing solutions to the differential equation $(\mathcal{A}-\lambda)\,f=0$ by $\psi_{\lambda} > 0$ and $\varphi_{\lambda} > 0$.

The differential operator $\lambda-\mathcal{A}$ has an inverse operator called the resolvent $R_{\lambda}$ defined by

\begin{equation*}(R_{\lambda}\,f)(x)=\mathbb{E}_x \bigg[ \int_0^{\tau}{\rm e}^{-\lambda s} f(X_s) {\rm d} s \bigg]\end{equation*}

for all $x \in \mathbb{R}_+$ and functions $f \in \mathcal{L}_1^{\lambda}$, where $\mathcal{L}_1^{\lambda}$ is the set of functions f on $\mathbb{R}_+$ which satisfy the integrability condition $\mathbb{E}_x [ \int_0^{\tau}{\rm e}^{-\lambda s} |f(X_s)| {\rm d} s ] < \infty $. Here, $\tau$ is the first exit time from $\mathbb{R}_+$, i.e. $\tau = \inf \{ t \geq 0 \mid X_t \not \in \mathbb{R}_+ \}$. We also define the scale density of the diffusion by

\begin{equation*}S'(x) = \exp \bigg( - \int^x \frac{2 \mu(z)}{\sigma^2(z)}{\rm d} z \bigg),\end{equation*}

which is the (non-constant) solution to the differential equation $\mathcal{A}\,f=0$, and the speed measure of the diffusion by

\begin{equation*}m'(x) = \frac{2}{\sigma^2(x)S'(x)}.\end{equation*}

It is well known that the resolvent and the solutions $\psi_{\lambda}$ and $\varphi_{\lambda}$ are connected by the formula

(1) \begin{align} (R_{\lambda}f)(x) & = B^{-1}_{\lambda} \psi_{\lambda}(x)\int_x^{\infty}\varphi_{\lambda}(z)\,f(z)m'(z){\rm d} z \end{align}

\begin{align}&\qquad\qquad\quad + B^{-1}_{\lambda} \varphi_{\lambda}(x)\int_0^{x}\psi_{\lambda}(z)\,f(z)m'(z){\rm d} z, \nonumber\end{align}

where

\begin{equation*}B_{\lambda} = \frac{\psi_{\lambda}^{\prime}(x)}{S'(x)}\varphi_{\lambda}(x)-\frac{\varphi_{\lambda}^{\prime}(x)}{S'(x)}\psi_{\lambda}(x)\end{equation*}

denotes the Wronskian determinant (see [Reference Borodin and Salminen10, p. 19]). We remark that the value of $B_{\lambda}$ does not depend on the state variable x because an application of the harmonicity properties of $\psi_\lambda$ and $\varphi_\lambda$ yields

\[\frac{{\rm d} B_{\lambda}(x)}{{\rm d} x} = 0.\]

In calculations, it is sometimes also useful to use the identity

(2) \begin{equation} \int_x^y \mu(z)m'(z){\rm d} z = \frac{1}{S'(y)}-\frac{1}{S'(x)}.\end{equation}

2.2. The control problem

We consider a control problem where the goal is to minimize the average cost per unit time, so that the controller is only allowed to control the underlying process at exogenously given times. These times are given as the arrival times of an independent Poisson process, called the signal process or constraint, and thus the interarrival times are exponentially distributed.

More precisely, the set of admissible controls $\mathcal{Z}$ is given by those non-decreasing left-continuous processes $\zeta_{t \geq 0}$ that have the representation

\[\zeta_t=\int_{[0,t)} \eta_s {\rm d} N_s,\]

where N is the signal process and the integrand $\eta$ is $\{\mathcal{F}_t\}_{t \geq 0}$-predictable. The controlled dynamics are then given by the Itô integral

\begin{equation*}X^{\zeta}_t = X_0 + \int_0^{\tau_0^\zeta \wedge t} \mu(X^{\zeta}_s) {\rm d} s + \int_0^{\tau_0^\zeta \wedge t} \sigma(X^{\zeta}_s){\rm d} W_s - \zeta_t, \qquad 0 \leq t \leq \tau_0^\zeta,\end{equation*}

where $\tau_0^\zeta$ is the first exit time of $X^{\zeta}_t$ from $\mathbb{R}_+$, i.e. $\tau_0^\zeta = \inf \{ t \geq 0 \mid X^{\zeta}_t \not \in \mathbb{R}_+ \}$.

Define the average cost per unit time or ergodic cost criterion as

\begin{equation*}J(x,\zeta)\coloneqq \liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x \left[ \int_0^{T} (\pi(X^{\zeta}_s){\rm d} s + \gamma {\rm d} \zeta_s) \right]\!,\end{equation*}

where $\gamma$ is a given positive constant and $\pi\,{:}\,\mathbb{R}_+ \to \mathbb{R}$ is a function measuring the cost of continuing the process. Now define the value function

(3) \begin{equation} V(T,x) = \inf_{\zeta \in \mathcal{Z}} \mathbb{E}_x \left[ \int_0^T (\pi (X_s){\rm d} s + \gamma {\rm d} \zeta_s) \right]\end{equation}

and denote by $\beta$ the minimum average cost. The objective of the control problem is to minimize $J(x, \zeta)$ over all the admissible controls $\zeta \in \mathcal{Z}$ and to find, if possible, the optimal control $\zeta^*$ such that $\beta = \inf_{\zeta \in \mathcal{Z}} J(x,\zeta) = J(x,\zeta^*)$.

We now define the auxiliary functions $\pi_{\gamma}\,{:}\,\mathbb{R}_+ \to \mathbb{R}$,

\begin{equation*}\pi_{\gamma}(x) = \pi(x)+ \gamma \lambda x ,\end{equation*}

and $\pi_{\mu}\,{:}\,\mathbb{R}_+ \to \mathbb{R}$,

\begin{equation*}\pi_{\mu}(x) = \pi(x)+\gamma \mu(x).\end{equation*}

In order for our solution to be well behaved, we must pose the following assumptions:

The boundaries of the state space are assumed to be natural, which means that, in the absence of interventions, the process cannot become infinitely large or infinitely close to zero in finite time. In biological applications these boundary conditions guarantee that the population does not explode or become extinct in the absence of harvesting. We refer to [Reference Borodin and Salminen10, pp. 18--20] for a more thorough discussion of the boundary behaviour of one-dimensional diffusions. Also, it is worth mentioning that no second-order properties of $\pi_{\mu}$ are assumed.

In addition, the following limiting and integrability conditions on the scale density and speed measure must be satisfied. These conditions assure the existence of a stationary distribution of the underlying diffusion.

2.3. Auxiliary results

Define the auxiliary functions $L\,{:}\, \mathbb{R_+} \to \mathbb{R}$ and $H\,{:}\, \mathbb{R_+} \to \mathbb{R}$ as

\begin{align*} L(x)&= \lambda \int_x^\infty \pi_{\mu}(z)\varphi_{\lambda}(z)m'(z){\rm d} z + \frac{\varphi^{\prime}_\lambda(x)}{ S'(x)}\pi_\mu(x), \\[3pt] H(0,x) &= \int_{0}^x \pi_{\mu}(z)m'(z){\rm d} z - \pi_{\mu}(x)m(0,x).\end{align*}

These functions will offer a convenient representation of the optimality equation in Section 3, and thus their properties play a key role when determining the optimal control policy.

Proof. The proof of the claim on L is similar to that of [Reference Lempa26, Lemma 3.3]. However, to show that the results are in accordance with Remark 1, we need to adjust the argument on finding a point $x_1 < x^*$ such that $L(x_1)<0$. Thus, assume for a while that $\lim_{x \to 0} \pi_{\mu}(x) = \infty$, and that $x^* > y > x$. Then

\begin{align*}L(x)-L(y) & = \lambda \int_x^y \pi_\mu(z) \varphi_\lambda (z) m'(z) {\rm d} z \\[3pt] & \quad + \bigg[ \pi_{\mu}(x)\frac{\varphi^{\prime}_\lambda(x)}{S'(x)}-\pi_{\mu}(y)\frac{\varphi^{\prime}_\lambda(y)}{ S'(y)} \bigg] \\[3pt]& \leq \frac{\varphi^{\prime}_\lambda(y)}{S'(y)}(\pi_\mu(x)-\pi_\mu(y)),\end{align*}

which shows that $\lim_{x \to 0} L(x) = - \infty$.

To prove the second part, assume first that $y>x>x^*$. Since the function $\pi_\mu$ is increasing on $(x^*,\infty)$, we see that

\begin{align*} H(0,y)-H(0,x) & = \int_{x}^y \pi_\mu(z)m'(z){\rm d} z - \pi_\mu(y)m(0, y)+\pi_\mu(x)m(0, x) \\[2pt]& < \pi_\mu(y)(m(x,y)-m(0, y)) +\pi_\mu(x)m(0, x) \\[2pt]& = m(0, x)(\pi_\mu(x) - \pi_\mu(y)) \\[2pt]& <0,\end{align*}

proving that H is decreasing on $(x^*, \infty)$. It also follows from

\[H(0,y)-H(0,x) < m(0,x)(\pi_\mu(x)-\pi_\mu(y)) \]

that $\lim_{y \to \infty} H(0,y) < 0$. Next, assume that $x^* > y > x$. Because $\pi_\mu$ is decreasing on $(0, x^*)$, we find similarly that

\begin{align*}H(0,y)-H(0,x)& = \int_{x}^y \pi_\mu(z)m'(z){\rm d} z - \pi_\mu(y)m(0, y)+\pi_\mu(x)m(0, x) \\[2pt]& > \pi_\mu(y)(m(x,y)-m(0, y)) +\pi_\mu(x)m(0, x) \\[2pt]& = m(0, x)(\pi_\mu(x) - \pi_\mu(y)), \\[2pt]& > 0,\end{align*}

implying that H is increasing on $(0, x^*)$. Furthermore, H is positive when $x < x^*$. Hence, by continuity, H has a unique root, which we denote by $\hat{x}$.□

Proposition 1. There exists a unique solution $\bar{x} \in (\tilde{x}, \hat{x})$ to the equation

\begin{equation*} S'(x) m(0,x) L(x) = -\varphi_{\lambda}^{\prime}(x)H(0,x).\end{equation*}

Proof. Define the function

\begin{equation*}P(x)= S'(x) m(0,x) L(x) + \varphi_{\lambda}^{\prime}(x)H(0,x).\end{equation*}

Assuming that $x_1 > \hat{x} > x^*$, we get, by Lemma 1, that

\begin{align*}P(x_1) = & S'(x_1) m(0,x_1) L(x_1) + \varphi_{\lambda}^{\prime}(x_1)H(0,x_1) \geq 0.\end{align*}

Similarly, when $x_2 < \tilde{x} < x^*$ we have that

(4) \begin{align} P(x_1) = & S'(x_2) m(0,x_2) L(x_2) + \varphi_{\lambda}^{\prime}(x_2)H(0,x_2) \leq 0.\end{align}

By continuity, the function P(x) must have at least one root. We denote one of these roots by z.

To prove that the root z is unique, we first notice that the naturality of the upper boundary implies that [Reference Borodin and Salminen10, p. 19]

\[\lim_{x \to \infty} \frac{\varphi_{\lambda}^{\prime}(x)}{S'(x)}=0.\]

Hence,

(5) \begin{align} -\frac{1}{\lambda} \frac{\varphi_{\lambda}^{\prime}(y^*)}{S'(y^*)} = \int_{y^*}^{\infty} \varphi_{\lambda}(z)m'(z){\rm d} z.\end{align}

Thus, we see that the equation $P(x)=0$ is equivalent to

\begin{equation*} \frac{\int_{x}^{\infty} \pi_{\mu}(y) \varphi_{\lambda}(y)m'(y){\rm d} y}{\int_{x}^{\infty} \varphi_{\lambda}(y)m'(y){\rm d} y} = \frac{\int_0^x \pi_{\mu}(y)m'(y){\rm d} y}{\int_0^x m'(y){\rm d} y}.\end{equation*}

Now, differentiating the left-hand side yields

\begin{equation*}\frac{\varphi_\lambda (x) m'(x) L(x)}{I(x)^2},\end{equation*}

where $I(x) = \int_{x}^{\infty} \varphi_{\lambda}(y)m'(y){\rm d} y$. Differentiating the right-hand side and evaluating it at z, we get, using the equation $P(z)=0$,

\begin{equation*} \frac{\pi_{\mu}(z)m'(z)}{m(0,z)}-\frac{\int_0^z \pi_{\mu}(y)m'(y){\rm d} y}{m(0,z)}\frac{m'(z)}{m(0,z)} = \frac{-m'(z)L(z)}{I(z)m(0,z)}.\end{equation*}

Because $L(y)>0$ in the region $(\tilde{x}, \hat{x})$, and all the other terms are positive everywhere, by comparing the derivatives we find that

\begin{equation*} \frac{-m'(z)L(z)}{I(z)m(0,z)} < \frac{\varphi_\lambda (z) m'(z) L(z)}{I(z)^2}.\end{equation*}

Therefore, by continuity, the intersection between the curves

\[I(x)^{-1}\int_{x}^{\infty} \pi_{\mu}(y) \varphi_{\lambda}(y)m'(y){\rm d} y\]

and

\[ m(0,x)^{-1}\int_0^x \pi_{\mu}(y)m'(y){\rm d} y \]

is unique. This unique point is denoted by $\bar{x}$. □

In the next lemma we make some further computations that are needed for the sufficient conditions of the control problem. Define the functions $J\,{:}\,\mathbb{R}_+ \to \mathbb{R}$ and $I\,{:}\,\mathbb{R}_+ \to \mathbb{R}$ as

\begin{equation*}J(x) = \frac{\gamma - (R_{\lambda} \pi_{\gamma})'(x)}{\varphi'_{\lambda}(x)}, \qquad I(x) = \frac{\int_{0}^x \pi_{\mu}(x) m'(t) {\rm d} t}{m(0,x)}.\end{equation*}

Here, $\tilde{x}$ and $\hat{x}$ are as in Lemma 1.

Proof. The first claim follows from the formula

\begin{equation*}J'(x) = \frac{2S'(x)}{\sigma^2(x)\varphi'_{\lambda}(x)^2} L(x) ,\end{equation*}

which can be derived using representation (1) and straightforward differentiation (see [Reference Lempa26, Lemma 3] for details). The claim on I follows similarly, as differentiation yields

\begin{align*} I'(x) = -\frac{m'(x)}{m^2(0,x)} H(x). \\[-30pt]\end{align*}

□

3.1. Necessary conditions

Denote the candidate solution for (3) as F(T, x). We use the heuristic that F(T, x) can be separated for large T as

(6) \begin{equation}F(T,x) \sim \beta T + W(x).\end{equation}

In mathematical finance literature, the constant $\beta$ usually denotes the minimum average cost per unit time and W(x) is the potential cost function (see [Reference Fleming15, Reference Sethi, Zhang and Zhang37]). The fact that the leading term $\beta T$ is independent of x is, of course, dependent on the ergodic properties of the underlying process. We also note that this heuristic can be used as a separation of variables to solve a partial differential equation of parabolic type related to the expectation in (3) via the Feynman–Kac formula [Reference Fleming and McEneaey16].

We shall proceed as in [Reference Lempa26]. We assume that the optimal control policy exists and is given by the following. When the process is below some threshold $y^*$ (called the waiting region) we let the process run, but if the process is above the threshold value $y^*$ (called the action region) and the Poisson process jumps we exert the exact amount of control to push the process back to the boundary $y^*$ and start it anew.

In the waiting region $[0,y^*]$ we expect that the candidate solution satisfies Bellman’s principle,

\begin{equation*}F(T,x)=\mathbb{E}\left[ \int_0^U \pi (X_s){\rm d} s + F(T-U,X_U) \right]\!,\end{equation*}

where U is an exponentially distributed random variable with mean $1/\lambda$.

Using the heuristic (6) and noticing the connection between the random times U and the resolvent, we get, by independence and the strong Markov property, that

\begin{align*}& \mathbb{E}\left[ \int_0^U \pi (X_s){\rm d} s + F(T-U,X_U) \right] \\ &= \lim_{r \to 0}(R_r \pi)(x) + \mathbb{E}_x [W(X_U)] - \frac{\beta}{\lambda} + \beta T -\mathbb{E}_x\left[ \int_U^{\infty} \pi (X_s){\rm d} s \right] \\ &= \lim_{r \to 0}(R_r \pi)(x) + \lambda (R_{\lambda}W)(x) - \frac{\beta}{\lambda} + \beta T -\mathbb{E}_x\left[ \lim_{r \to 0}(R_r \pi)(X_U) \right] \\ &= \lim_{r \to 0}(R_r \pi)(x) + \lambda (R_{\lambda}W)(x) - \frac{\beta}{\lambda} + \beta T - \lambda \lim_{r \to 0}(R_{\lambda} R_r \pi)(x).\end{align*}

Hence, we arrive at the equation

\begin{equation*}W(x) - \lim_{r \to 0}(R_r \pi)(x) = \lambda R_{\lambda}(W(x) - \lim_{r \to 0}(R_r\pi)(x)) - \frac{\beta}{\lambda} .\end{equation*}

We next choose $f(x) = W(x) - \lim_{r \to 0}(R_r \pi)(x)$ in [Reference Lempa24, Lemma 2.1], and notice that the lemma remains unchanged even if we add a constant $\beta / \lambda$. Thus, we expect, by our heuristic arguments, that the pair $(W, \beta)$ satisfies the differential equation

\begin{equation*}\mathcal{A}W(x) + \pi(x)= \beta.\end{equation*}

This type of equation often arises in ergodic control problems and there is lots of literature on sufficient conditions for the existence of a solution [Reference Arapostathis, Borkar and Ghosh7, Reference Fleming15, Reference Robin34]. We remark here that usually these conditions rely heavily on the solution of the corresponding discounted infinite-horizon control problem, and thus apply the so-called vanishing discount method. However, in our case we can proceed by explicit calculations.

Next, we shall determine the equation for the pair $(W, \beta)$ in the action region $[y^*, \infty]$. The Poisson process jumps in infinitesimal time with probability $\lambda {\rm d} t$, and in that case the agent has to pay a cost $\gamma (x- y^*) + F(T, y^*)$. On the other hand, the Poisson process does not jump with probability $1- \lambda {\rm d} t$, and in this case the agent has to pay $\pi(x) {\rm d} t + \mathbb{E}_x\left[ F(T-{\rm d} t,X_{{\rm d} t}) \right]$. Thus, the candidate function F should satisfy the condition

\begin{equation*}F(T,x) = \lambda {\rm d} t(\gamma (x- y^*) + F(T, y^*)) + (1- \lambda {\rm d} t)\left( \pi(x) {\rm d} t + \mathbb{E}_x\left[ F(T-{\rm d} t,X_{{\rm d} t}) \right] \right)\!.\end{equation*}

Now, again using the heuristic (6) and that, intuitively, ${\rm d} t^2=0$, we find that

\[W(x) = \lambda {\rm d} t (\gamma (x-y^*) + W(y^*)) + \pi(x){\rm d} t - \beta {\rm d} t + (1- \lambda {\rm d} t)\mathbb{E}_x \left[ W(X_{{\rm d} t}) \right]\!.\]

By formally using Dynkin’s formula on the last term and simplifying we get

\[0 = \lambda {\rm d} t (\gamma (x-y^*) + W(y^*)) + \pi(x){\rm d} t - \beta {\rm d} t + (\mathcal{A} - \lambda)W(x) {\rm d} t.\]

We conclude that in the action region, the pair $(W, \beta)$ should satisfy the differential equation

\begin{equation*} (\mathcal{A} - \lambda)W(x) = -(\pi(x) + \lambda (\gamma (x-y^*) + W(y^*))-\beta).\end{equation*}

Now, we first observe that

\begin{equation*}\mathcal{A}W(x)+\pi(x)-\beta = \begin{cases} 0, & x<y^*,\\ \lambda(W(x)- \gamma x -(W(y^*)-\gamma y^*)), & x\geq y^*, \end{cases}\end{equation*}

which implies that W(x) satisfies the $C^1$-condition $W'(y^*)=\gamma$. We have thus arrived at the following free boundary problem: find a function W(x) and constants $y^*$ and $\beta$ such that

(7) \begin{align} W & \in C^2, \nonumber \\[3pt] W'(y^*) & = \gamma, \nonumber \\[3pt] (\mathcal{A} - \lambda)W(x) + \pi(x) + \lambda (\gamma (x-y^*) + W(y^*)) & = \beta, \qquad x \in [y^*, \infty), \end{align}

(8) \begin{align}\hspace*{116pt}\mathcal{A}W(x) + \pi(x) & = \beta, \qquad x \in (0,y^*). \end{align}

Remark 2. Another common approach to heuristically form the Hamilton–Jacobi–Bellman (HJB) equation of the problem is to use the value function $J_r(x)$ of the corresponding discounted problem (see [Reference Lempa26, p. 4]) and the vanishing discount limits $rJ_r(\bar{x}) \to \beta$ and $W_r(x) = J_r(x)-J_r(\bar{x}) \to W(x)$, where $\bar{x}$ is a fixed point in $\mathbb{R}_+$ (see [Reference Robin34, p. 284] and [Reference Fleming15, p. 427]). This argument yields the HJB equation

\begin{align*} \mathcal{A}W(x) + \pi(x) - \lambda(W'(x)-\gamma)\textbf{1}_{\{ x \in S \}} = \beta,\end{align*}

where $S = [y^*,\infty)$ is the control region.

To solve the free boundary problem, we consider (8) first. In this case we write the differential operator $\mathcal{A}$ as [Reference Revuz and Yor38, p. 285]

\[\mathcal{A} = \frac{{\rm d}}{{\rm d} m(x)} \frac{{\rm d}}{{\rm d} S(x)},\]

which allows us to find that

(9) \begin{equation} \frac{{\rm d} W'(x)}{{\rm d} S'(x)} = (\beta - \pi (x))m'(x).\end{equation}

Therefore, integrating over the interval $(0, y^*)$ gives

\begin{equation*}\frac{W'(y^*)}{S'(y^*)} = \beta \int_{0}^{y^*} m'(z){\rm d} z - \int^{y^*}_0 \pi(z)m'(z) {\rm d} z.\end{equation*}

Hence, by Assumption 3 and the $C^1$-condition $W'(y^*)=\gamma$, we get

\begin{equation*}\beta = \bigg[\int_{0}^{y^*} m'(z){\rm d} z\bigg]^{-1} \left[ \int^{y^*}_{0} \pi(z)m'(z) {\rm d} z + \frac{\gamma}{S'(y^*)} \right]\!.\end{equation*}

Finally, using (2), we arrive at

(10) \begin{equation} \beta = \bigg[\int_{0}^{y^*} m'(z){\rm d} z\bigg]^{-1} \left[ \int^{y^*}_{0} \pi_{\mu}(z)m'(z) {\rm d} z \right]\!.\end{equation}

Next, we consider (7). We immediately find that a particular solution is

\begin{equation*}W(x) = (R_{\lambda} \pi_{\gamma})(x) - \frac{\beta}{\lambda} - \gamma y^* + W(y^*).\end{equation*}

Hence, we conjecture analogously to [Reference Lempa26, p. 113] that the solution to (7) is

(11) \begin{equation} W(x) = (R_{\lambda} \pi_{\gamma})(x) - \frac{\beta}{\lambda} - \gamma y^* + W(y^*) + C \varphi_{\lambda}(x).\end{equation}

To find the constants C and $\beta$, we first use the continuity of W at the boundary $y^*$, which allows us to substitute $x=y^*$ into (11). This yields

(12) \begin{equation} 0 = (R_{\lambda} \pi_{\gamma})(y^*) - \frac{\beta}{\lambda} - \gamma y^* + C\varphi_{\lambda}(y^*).\end{equation}

Then, by applying the condition $W'(y^*) = \gamma$ to (11), we find that

\begin{equation*}C = \frac{\gamma - (R_{\lambda} \pi_{\gamma})'(y^*)}{\varphi_{\lambda}^{\prime}(y^*)}.\end{equation*}

Combining this with (12) gives

\begin{equation*}\beta = \lambda (R_{\lambda} \pi_{\gamma})(y^*)- \lambda \gamma y^* + \frac{\gamma - (R_{\lambda} \pi_{\gamma})'(y^*)}{\varphi_{\lambda}^{\prime}(y^*)} \lambda \varphi_{\lambda}(y^*).\end{equation*}

To rewrite this expression, we first notice that a straightforward differentiation gives

\begin{align*}\frac{{\rm d}}{{\rm d} x} \bigg( x \frac{\varphi_{\lambda}^{\prime}(x)}{S'(x)} - \frac{\varphi_{\lambda}(x)}{S'(x)} \bigg) = -m'(x)\varphi_{\lambda}(x)(\mu(x)- \lambda x).\end{align*}

Thus, by the fundamental theorem of calculus and the naturality of the upper boundary, we get

(13) \begin{equation} y^* \frac{\varphi_{\lambda}^{\prime}(y^*)}{S'(y^*)} - \frac{\varphi_{\lambda}(y^*)}{S'(y^*)} = \int_{y^*}^{\infty} m'(z)\varphi_{\lambda}(z)(\mu(z)- \lambda z) {\rm d} z.\end{equation}

Next, using (1), we find that

\begin{equation*} (R_{\lambda} \pi_{\gamma})(y^*)\varphi_{\lambda}^{\prime}(y^*) - (R_{\lambda} \pi_{\gamma})'(y^*) \varphi_{\lambda}(y^*) = - S'(y^*)\int_{y^*}^{\infty} \varphi_{\lambda}(z) \pi_{\gamma}(z)m'(z){\rm d} z.\end{equation*}

Combining these observations with (5), the constant $\beta$ reads

\begin{equation*}\beta = \bigg[ \int_{y^*}^{\infty} \varphi_{\lambda}(z)m'(z){\rm d} z \bigg]^{-1} \bigg[ \int_{y^*}^{\infty} \!\! \varphi_{\lambda}(z) \pi_{\gamma}(z)m'(z){\rm d} z + \gamma \int_{y^*}^{\infty} \!\! m'(z)\varphi_{\lambda}(z)(\mu(z)- \lambda z) {\rm d} z \bigg]. \end{equation*}

Finally, by recalling the definition of $\pi_{\mu}(x)$, we have

(14) \begin{equation} \beta= \bigg[ \int_{y^*}^{\infty} \varphi_{\lambda}(z)m'(z){\rm d} z \bigg]^{-1} \bigg[ \int_{y^*}^{\infty} \varphi_{\lambda}(z) \pi_{\mu}(z)m'(z){\rm d} z \bigg].\end{equation}

Now, by equating the representations (10) and (14) of $\beta$, we find the optimality condition

\begin{equation*} \int^{y^*}_{0} \pi_{\mu}(z)m'(z) {\rm d} z \int_{y^*}^{\infty} \varphi_{\lambda}(z)m'(z){\rm d} z = \int_{0}^{y^*} m'(z){\rm d} z \int_{y^*}^{\infty} \varphi_{\lambda}(z) \pi_{\mu}(z)m'(z){\rm d} z,\end{equation*}

which can be re-expressed, using the functions L(x) and H(0, x), as

(15) \begin{equation} m(0,y^*)L(y^*) = -\varphi_{\lambda}^{\prime}(y^*)H(0,y^*).\end{equation}

We proved in Proposition 1 that there exists a unique solution $\bar{x}$ to (15), and thus we will assume in the following that $y^*=\bar{x}$.

Remark 3. As often in ergodic optimal control problems, the potential value function W(x) satisfies second-order differentiability across the boundary $\lim_{x \downarrow y^*} W''(x)= \lim_{x \uparrow y^*} W''(x)$. This can be verified as follows. When $x>y^*$, the differentiation and the harmonicity properties $(R_{\lambda} (\mathcal{A-\lambda})\pi_{\gamma})(x)+\pi_{\gamma}(x) = 0$ and $(\mathcal{A}-\lambda)\varphi_{\lambda}(x) = 0$ yield

\begin{align*} \lim_{x \downarrow y^*}W''(x) & = (R_{\lambda} \pi_{\gamma})''(y^*) + C \varphi_{\lambda}^{\prime\prime}(y^*) \\[3pt] & = \frac{2}{\sigma^2(y^*)} \bigg[ \lambda(R_{\lambda} \pi_{\gamma})(y^*) - \pi_{\gamma}(y^*) - \mu(y^*)(R_{\lambda} \pi_{\gamma})'(y^*) \\[3pt] & \qquad\qquad\ \ + \frac{\gamma-(R_{\lambda} \pi_{\gamma})'(y^*)}{\varphi_{\lambda}^{\prime}(y^*)}(\lambda \varphi_{\lambda}(y^*)- \mu(y^*)\varphi_{\lambda}^{\prime}(y^*)) \bigg], \end{align*}

which after cancellation and applying (13) and (1) equals

\begin{align*} \frac{2}{\sigma^2(y^*)} \bigg[ -\frac{\lambda S'(y^*)}{\varphi_{\lambda}^{\prime}(y^*)} \int_{y^*}^{\infty} \varphi_{\lambda}(z) m'(z) ( \pi_{\gamma}(z) - \gamma \lambda z + \mu(z) \gamma){\rm d} z - \pi_{\mu}(y^*) \bigg].\end{align*}

Therefore, by using (5) and (14) we find that

\begin{align*} \lim_{x \downarrow y^* }W''(x) = \frac{2}{\sigma^2(y^*)} [ \beta - \pi_{\mu}(y^*) ].\end{align*}

On the other hand, when $x < y^*$ we notice that

\begin{equation*} \frac{{\rm d}}{{\rm d} x} \bigg[ \frac{W'(x)-\gamma}{S'(x)} \bigg] = (\mathcal{A}W'(x) - \gamma \pi(x))m'(x) = (\beta - \pi_{\mu}(x))m'(x).\end{equation*}

Hence, by differentiating the left-hand side and plugging in $x = y^*$, we find by the first-order condition $W'(y^*) = \gamma$ that

\begin{equation*} \lim_{x \uparrow y^* }W''(x) = \frac{2}{\sigma^2(y^*)} [ \beta - \pi_{\mu}(y^*) ].\end{equation*}

3.2. Sufficient conditions

We begin with an initial remark. When $x > y^*$, we get, by differentiating (11) and using Lemma 2, that

\begin{equation*} W'(x)-\gamma = \varphi'_{\lambda}(x)\bigg[ \frac{(R_{\lambda} \pi_{\gamma})'(x)-\gamma}{\varphi'_{\lambda}(x)} -\frac{(R_{\lambda} \pi_{\gamma})'(y^*)-\gamma}{\varphi'_{\lambda}(y^*)} \bigg] > 0.\end{equation*}

In the opposite case, when $x < y^*$, we have

\begin{equation*} \frac{{\rm d}}{{\rm d} x} \bigg[ \frac{W'(x)-\gamma}{S'(x)} \bigg] = (\beta - \pi_{\mu}(x))m'(x).\end{equation*}

Thus, by integrating over the interval (0, x), and using (10) and Lemma 2, we find that

\begin{equation*} \frac{W'(x)-\gamma}{S'(x)} = m(0, x) \bigg[ \frac{\int_{0}^{y^*} \pi_{\mu}(t)m'(t){\rm d} t }{m(0, y^*)} - \frac{\int_{0}^{x} \pi_{\mu}(t)m'(t){\rm d} t }{m(0, x)} \bigg] < 0.\end{equation*}

These observations imply that, under the standing assumptions, the function $W(x)-\gamma x$ has a global minimum at $y^*$, which shows that W(x) satisfies the variational equality

(16) \begin{equation} \mathcal{A}W(x)+\pi(x) + \lambda \Big[ \inf_{y \leq x}\{(W(y)-\gamma y)-(W(x)-\gamma x)\} \Big] = \beta.\end{equation}

Proposition 2. (Verification.) Under Assumptions 1, 2, and 3, the optimal policy is as follows. If the controlled process $X^{\zeta}$ is above the threshold $y^*$ at a jump time of N, i.e. $X^{\zeta}_{T_-} > y^*$, the decision maker should take the controlled process $X^{\zeta}$ to $y^*$. Further, the threshold $y^*$ is uniquely determined by (15), and the constant $\beta$ characterized by (10) and (14) gives the minimum average cost,

\begin{equation*} \beta = \inf_{\zeta} \liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x \bigg[ \int_0^T (\pi(X_s){\rm d} s + \gamma {\rm d} \zeta_s) \bigg].\end{equation*}

Proof. Define the function

\begin{align*} \Phi(x) & \coloneqq \inf_{y \leq x}\{(W(y)-\gamma y)-(W(x)-\gamma x)\} \\[3pt] & = \{W(y^*)-W(x)+\gamma(x-y^*) \} \textbf{1}_{[y^*,\infty)}(x),\end{align*}

and a family of almost surely finite stopping times $\tau(\rho)_{\rho > 0}$ as $\tau(\rho)\coloneqq \tau_0^\zeta \wedge \rho \wedge \tau^\zeta_{\rho}$, where $\tau_{\rho}^\zeta = \inf \{t \geq 0\,{:}\, X_t^\zeta \not \in (1/\rho, \rho) \}$. By applying the Doléans–Dade–Meyer change of variables formula to the process $W(X_t)$, we obtain

\begin{align*} W(X_{t \wedge \tau(\rho)}) - W(x) = & \int_0^{t \wedge \tau(\rho)} \mathcal{A}W\big(X_s^{\zeta}\big){\rm d} s + \int_0^{t \wedge \tau(\rho)} \sigma\big(X_s^{\zeta}\big)W'\big(X_s^{\zeta}\big){\rm d} B_s \\[3pt] & + \sum_{0 \leq s \leq {t \wedge \tau(\rho)}} \big[W\big(X_s^{\zeta}\big)-W\big(X_{s-}^{\zeta}\big)\big].\end{align*}

Because the control $\zeta$ jumps only if the Poisson process N jumps, we have that $W(X_s^{\zeta}) - W(X_{s-}^{\zeta}) + \gamma(\Delta \zeta_s) \geq \Phi(X_{s-}^{\zeta})$. By combining these two observations with (16), we get

(17) \begin{equation} W(X_{t \wedge \tau(\rho)}) \geq W(x) + \beta ({t \wedge \tau(\rho)}) - \int_0^{t \wedge \tau(\rho)} \big[\pi\big(X_s^{\zeta}\big){\rm d} s + \gamma {\rm d} \zeta_s\big] + Z_{t \wedge \tau(\rho)} + M_{t \wedge \tau(\rho)}, \end{equation}

where

\begin{equation*} M_t \coloneqq \int_0^t \sigma\big(X_s^{\zeta}\big)W'\big(X_s^{\zeta}\big){\rm d} B_s, \quad \quad Z_t \coloneqq \int_0^t \Phi\big(X_s^{\zeta}\big){\rm d} \tilde{N}_s.\end{equation*}

Here, $\tilde{N}_t = (N_t-\lambda t)_{t \geq 0}$ is the compensated Poisson process. It follows from the calculation above that $Z_{t \wedge \tau(\rho)} + M_{t \wedge \tau(\rho)}$ is a submartingale and thus $\mathbb{E}_x [ Z_{t \wedge \tau(\rho)} + M_{t \wedge \tau(\rho)} ] \geq 0$. Taking expectation on both sides, dividing by ${t \wedge \tau(\rho)}$, and letting $t, \rho \to \infty$, we find that

\begin{equation*} \liminf_{T\to \infty} \frac{1}{T} \mathbb{E}_x \bigg[ W\big(X_T^{\zeta}\big) + \int_0^T (\pi\big(X_s^{\zeta}\big) {\rm d} s + \gamma {\rm d} \zeta_s) \bigg] \geq \beta.\end{equation*}

Thus, if $\liminf_{T\to \infty} \frac{1}{T} \mathbb{E}_x [ W(X_T^{\zeta})] = 0$, it follows that $J(x,\zeta) \geq \beta$; we postpone the proof of this limiting property to the following lemma.

Next, we prove that ${J(x,\zeta^*) \leq \beta}$. We proceed as above and note that (17) holds as equality when $\zeta = \zeta^*$. Hence, the local martingale term $M_T+Z_T$ is now uniformly bounded from below by $-W(x)-\beta T$, and is therefore a supermartingale. Thus, we have

\begin{equation*} \mathbb{E}_x \bigg[\int_0^T \big(\pi\big(X_s^{\zeta}\big) {\rm d} s + \gamma {\rm d}\zeta^*_s\big) \bigg] \leq \beta T + W(x)- \mathbb{E}_x[W(X_T)] \leq \beta T + W(x).\end{equation*}

Finally, dividing by T and letting $T \to \infty$, we get $J(x,\zeta^*) \leq \beta$, which completes the proof. □

As is usual in ergodic control problems, we noticed in the proof that the verification theorem holds under the assumption that $\liminf_{T\to \infty} \frac{1}{T} \mathbb{E}_x [ W(X_T^{\zeta})] = 0$. Thus, in the following lemma we give the sufficient condition on $\pi(x)$ under which the limit equals zero.

Lemma 3. The limit

(18) \begin{equation} \liminf_{T\to \infty} \frac{1}{T} \mathbb{E}_x \big[ W\big(X_T^{\zeta}\big)\big] = 0\end{equation}

holds if $\pi(x) \geq C (x^{\alpha}-1)$, where $\alpha$ and C are positive constants.

Proof. Let $x > y^*$. Then W(x) reads

\begin{align*}W(x) & = (R_{\lambda} \pi_{\gamma})(x) - \frac{\beta}{\lambda} - \gamma y^* + W(y^*) + C \varphi_{\lambda}(x) , \\[3pt] W(x) & = (R_{\lambda} \pi_{\mu})(x) - \frac{\beta}{\lambda} + \gamma (x-y^*) + W(y^*) + C \varphi_{\lambda}(x).\end{align*}

Because $\varphi_{\lambda}(x)$ is bounded in this region, we only need to deal with the resolvent term. By the Markov property and the substitution $k=s+T$, we find that

\begin{align*} \mathbb{E}_x [(R_{\lambda} \pi_{\gamma})(X_T)] & = \int_0^{\infty}{\rm e}^{-\lambda s} \mathbb{E}_x \big[ \mathbb{E}_{X_T} \big[ \pi_{\gamma}(X_s) \big] \big] {\rm d} s \\[3pt] & = \int_0^{\infty}{\rm e}^{-\lambda s} \mathbb{E}_x \big[ \mathbb{E}_{x} \big[ \pi_{\gamma}(X_{s+T}) \mid \mathcal{F}_T \big] \big] {\rm d} s \\[3pt] & = \int_0^{\infty}{\rm e}^{-\lambda s} \mathbb{E}_x \big[ \pi_{\gamma}(X_{s+T}) \big] {\rm d} s \\[3pt] & = {\rm e}^{\lambda T} \int_T^{\infty}{\rm e}^{-\lambda k} \mathbb{E}_x \big[ \pi_{\gamma}(X_{k}) \big] {\rm d} k.\end{align*}

By l’Hôpital’s rule and the assumption that id and $\pi$ are elements of $\mathcal{L}^{\lambda}_1$, we find that

\begin{equation*} \liminf_{T \to \infty} \frac{{\rm e}^{\lambda T}}{T} \int_T^{\infty}{\rm e}^{-\lambda k} \mathbb{E}_x \big[ \pi_{\gamma}(X_{k}) \big] {\rm d} k = \liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_{x} \big[ \pi_{\gamma}(X_{T}) \big].\end{equation*}

On the other hand, if

\[\liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x [X_T] > 0,\]

there exists $T_1$ such that

\[\mathbb{E}_x [X_s] > \varepsilon \frac{s}{(\alpha+1)^{1/\alpha}}\]

for all $s > T_1$. Together with the assumption $\pi(x) \geq C (x^{\alpha}-1)$, this leads to contradiction as

\begin{align*} \infty & > \liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x \bigg[ \int_0^T (\pi(X_s){\rm d} s + \gamma {\rm d} \zeta_s) \bigg] \\ & \geq \liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x \bigg[ \int_0^T \pi(X_s){\rm d} s \bigg] \\ & \geq -C + C \liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x \bigg[ \int_0^T X_s^{\alpha} {\rm d} s \bigg] \\ & \geq -C + C \liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x \bigg[ \frac{\varepsilon^{\alpha}}{\alpha+1}\int_0^T s^{\alpha} {\rm d} s \bigg] \\ & = -C + C \varepsilon^{\alpha} \liminf_{T \to \infty} T^{\alpha} = \infty.\end{align*}

Similarly, we must have

\[\liminf_{T \to \infty} \frac{1}{T} \mathbb{E}_x [\pi(X_T)] = 0,\]

and thus conclude that the limit (18) must vanish.

In the opposite case $x < y^*$, we find by integrating in (9) that

\begin{equation*} \frac{W'(y)}{S'(y)} = \frac{\gamma}{S'(y^*)} + \int_y^{y^*} m'(z)(\pi(z)-\beta) {\rm d} z.\end{equation*}

Multiplying by S $^{\prime}(x)$ and integrating over the interval $(x,y^*)$, we have

\begin{equation*} W(x) = W(y^*) - \frac{\gamma}{S'(y^*)} \int_x^{y^*}S'(z){\rm d} z - \int_x^{y^*} \int_y^{y^*} m'(z)(\pi(z)-\beta) {\rm d} z S'(y){\rm d} y.\end{equation*}

Since the second term is negative and $\pi(x)$ is positive everywhere, this has the upper bound

\begin{equation*} W(x) \leq W(y^*) + \beta \int_x^{y^*} \int_y^{y^*} m'(z) {\rm d} z S'(y){\rm d} y.\end{equation*}

As the last integral is positive, we can by Assumption 3 expand the region of the inner integral to get

\begin{equation*} W(x) \leq W(y^*) + \beta \int_x^{y^*} \int_0^{y^*} m'(z) {\rm d} z S'(y){\rm d} y.\end{equation*}

Thus,

\begin{equation*} W(x) \leq W(y^*) + \beta m(0,y^*)(S(y^*)-S(x)).\end{equation*}

Consequently, the upper bound is of the form $W(x) \leq C_0 S(x) + C_1$, where $C_0$ and $C_1$ are constants. Since $S(X_t)$ is a non-negative local martingale [Reference Bass9, p. 88] and hence a supermartingale, we have

\begin{align*} \mathbb{E}_x[W(X_T)] \leq C_0 \mathbb{E}_x[S(X_T)] + C_1 & \leq C_0 \mathbb{E}_x[S(X_0)] + C_1 \\[3pt] & = C_0 S(x) + C_1.\end{align*}

Hence, also in this case the limit (18) must vanish. □

5.1. Verhulst–Pearl diffusion

We consider a standard Verhulst–Pearl diffusion

\[{\rm d} X_t = \mu X_t(1-\beta X_t){\rm d} t+\sigma X_t {\rm d} W_t, \qquad X_0 = x \in \mathbb{R}_r,\]

where $\mu >0$, $\sigma >0$, and $\beta > 0$. This diffusion is often used as a model for stochasticly fluctuating populations [Reference Alvarez and Hening4, Reference Evans, Hening and Schreiber13]. The scale density and speed measure in this case are

\[S'(x) = x^{-\frac{2 \mu}{\sigma^2}} {\rm e}^{\frac{2\mu \beta}{\sigma^2} x}, \qquad m'(x) = \frac{2}{\sigma^2} x^{\frac{2 \mu}{\sigma^2}-2} {\rm e}^{-\frac{2\mu \gamma}{\sigma^2} x}.\]

We assume that the cost $\pi(x) = x^2$ and $\gamma=1$. Hence, $\pi_{\mu}(x) = x^2 - x \mu (1-\beta x) $. In this setting, we note that if $\mu > \sigma^2/2$ then

\[m(0,x) = \frac{2}{\sigma^2}\bigg( \frac{\sigma^2}{2 \mu \beta}\bigg) ^{\frac{2 \mu}{\sigma^2}-1} \bigg( \bigg[ \Gamma \bigg( \frac{2\mu}{\sigma^2}-1 \bigg)- \Gamma \bigg( \frac{2\mu}{\sigma^2}-1,\frac{2 \mu \beta x}{\sigma^2} \bigg) \bigg] \bigg).\]

The minimal excessive functions read [Reference Dayanik and Karatzas11, pp. 201–203]

\begin{align*}\varphi_{\lambda}(x) & = x^{\alpha_1} U \bigg(\alpha_1, 1+ \alpha_1 -\alpha_2, \frac{2 \mu \beta x}{\sigma^2}\bigg), \\ \quad \psi_{\lambda}(x) & = x^{\alpha_1} M \bigg( \alpha_1, 1+\alpha_1-\alpha_2, \frac{2\mu \beta x}{\sigma^2} \bigg),\end{align*}

where U and M are Kummer’s confluent hypergeometric functions of the second and first kind respectively, and

\begin{equation*} \alpha_1 = \frac{1}{2}-\frac{\mu}{\sigma^2}+\sqrt{\bigg(\frac{1}{2}-\frac{\mu}{\sigma^2} \bigg)^2+\frac{2\lambda}{\sigma^2}}, \qquad \alpha_2 = \frac{1}{2}-\frac{\mu}{\sigma^2} - \sqrt{\bigg(\frac{1}{2}-\frac{\mu}{\sigma^2} \bigg)^2+\frac{2\lambda}{\sigma^2}}.\end{equation*}

We see that our assumptions are satisfied, and thus the result applies. Unfortunately, (15) for the optimal threshold $y^*$, and the formula for the minimum average cost $\beta$ (14), are complicated and therefore left unstated. However, we can illustrate the results numerically. In Table 1 the optimal threshold $y^*$ is calculated with the values $\mu = 1$, $\sigma=1$, and $\beta = 0.01$ for a few different values of $\lambda$. We see from the table that as $\lambda$ increases the threshold $y^*$ gets closer to the corresponding threshold $y^*_s \approx 0.743$ of the singular control problem (19).

Table 1: The values for the optimal threshold $y^*$ for some values of the intensity $\lambda$.

5.2. The standard Ornstein–Uhlenbeck process

As remarked in the introduction, the results also hold for $\mathbb{R}$ with straightforward changes. Indeed, we only have to adjust the assumptions slightly, by changing the lower boundary from 0 to $-\infty$ in Assumptions 2 and 3, and change all the formulas accordingly. With this change we can study a larger class of processes.

Consider dynamics that are characterized by the stochastic differential equation

\[{\rm d} X_t = - \beta X_t {\rm d} t + {\rm d} W_t, \qquad \, X_0=x,\]

where $\beta > 0$. This diffusion is often used to model continuous-time systems that have mean-reverting behaviour. To illustrate the results we choose the running cost $\pi(x) = |x|,$ and consequently $\pi_{\mu}(x) = |x|-\gamma \beta x$. The scale density and the density of speed measure in this case are

\[S'(x)= \exp\big( \beta x^2\big) , \qquad m'(x)=2 \exp\big(- \beta x^2\big),\]

and the minimal excessive functions read [Reference Borodin and Salminen10, pp. 141]

\[\varphi_{\lambda}(x) = {\rm e}^{\frac{\beta x^2}{2}} D_{-\lambda / \beta}\Big(x \sqrt{2 \gamma}\Big), \qquad \psi_{\lambda}(x) = {\rm e}^{\frac{\beta x^2}{2}} D_{-\lambda / \beta}\Big(-x \sqrt{2 \gamma}\Big),\]

where $D_{\nu}(x)$ is a parabolic cylinder function. Equation (15) for the optimal threshold again takes a rather complicated form and thus the results are only illustrated numerically in Table 2. In the singular control case (19) gives $y^*_s \approx 0.535$. Thus, as expected, the threshold value $y^*$ gets closer to $y^*_s$ when $\lambda$ increases.

Table 2: The value of the optimal threshold $y^*$ for a controlled Ornstein–Uhlenbeck process for $\beta=0.1$ and a few choices of $\lambda$.