Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-02-06T05:15:07.534Z Has data issue: false hasContentIssue false

Exponential ergodicity and steady-state approximations for a class of markov processes under fast regime switching

Published online by Cambridge University Press:  17 March 2021

Ari Arapostathis*
Affiliation:
The University of Texas at Austin
Guodong Pang*
Affiliation:
Pennsylvania State University
Yi Zheng*
Affiliation:
Pennsylvania State University
*
*Postal address: Department of Electrical and Computer Engineering, The University of Texas at Austin, 2501 Speedway, EERC 7.824, Austin, TX78712. Email address: ari@utexas.edu
**Postal address: The Harold and Inge Marcus Department of Industrial and Manufacturing Engineering, College of Engineering, Pennsylvania State University, University Park, PA16802.
**Postal address: The Harold and Inge Marcus Department of Industrial and Manufacturing Engineering, College of Engineering, Pennsylvania State University, University Park, PA16802.
Rights & Permissions [Opens in a new window]

Abstract

We study ergodic properties of a class of Markov-modulated general birth–death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that is taken to be large. Under very weak hypotheses, we show that if the averaged process is exponentially ergodic for large values of the parameter, then the same applies to the original joint Markov process. The second set of results concerns steady-state diffusion approximations, under the assumption that the ‘averaged’ fluid limit exists. Here, we establish convergence rates for the moments of the approximating diffusion process to those of the Markov-modulated birth–death process. This is accomplished by comparing the generator of the approximating diffusion and that of the joint Markov process. We also provide several examples which demonstrate how the theory can be applied.

Type
Original Article
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of Applied Probability Trust

1. Introduction

There has been a considerable amount of research on Markov-modulated birth–death processes. The rate control problem for Markov-modulated single-server queues has been addressed in [Reference Fralix and Adan10, Reference Kumar, Lewis and Topaloglu18, Reference Xia, He and Alfa24], while the scheduling control problem for Markov-modulated critically loaded multiclass many-server queues has been considered in [Reference Arapostathis, Das, Pang and Zheng3], in which exponential ergodicity under a static priority rule is also studied. The papers [Reference Anderson, Blom, Mandjes, Thorsdottir and De Turck1, Reference Jansen, Mandjes, De Turck and Wittevrongel14] address functional limit theorems for Markov-modulated Markovian infinite-server queues. See also the work on the functional limit theorem for Markov-modulated compound Poisson processes in [Reference Pang and Zheng22]. We refer the reader to [Reference Jonckheere and Shneer15, Reference Zeifman25] for the study of stability and instability for birth–death processes.

In this paper, we study a class of general birth–death processes with countable state space and bounded jumps. Meanwhile, the transition rate functions of the birth–death process depend on an underlying continuous-time Markov process with finite state space. An asymptotic framework is considered under which the Markov-modulated birth–death process is indexed by a scaling parameter n, with n getting large. The transition rate matrix of the underlying Markov process is of order $n^{\alpha}$ , $\alpha>0$ , and the jump size of the birth–death process shrinks at a rate of $n^{\beta}$ with $\beta:= \max\{1/2,1 -\alpha/2\}$ . This scaling has been used in [Reference Anderson, Blom, Mandjes, Thorsdottir and De Turck1, Reference Arapostathis, Das, Pang and Zheng3, Reference Jansen, Mandjes, De Turck and Wittevrongel14] for some special birth–death queueing processes.

In this asymptotic framework, we first provide a sufficient condition for the scaled Markov-modulated process to be exponentially ergodic. We show that if the ‘averaged’ birth–death process satisfies a Foster–Lyapunov criterion for a certain class of Lyapunov functions, then the original Markov-modulated process also has the same property. Next, we study steady-state approximations of the Markov-modulated process. We construct diffusion models, and show that their steady-state moments approximate those of the joint Markov process with a rate $n^{- (1/2\wedge \alpha/2)}$ . This problem is motivated by [Reference Gurvich11], in which steady-state approximations for a general birth–death process have been considered. However, the problem in this paper is quite challenging, since we need to consider the variabilities of the underlying Markov process, and the martingale argument in the above-referenced work cannot be applied. We also present some examples from queueing systems and show that the assumptions presented are easy to verify.

The aforementioned result of exponential ergodicity is stated in Theorem 2.1. We consider a large class of scaled Markov-modulated general birth–death processes, whose transition rate functions have linear growth around some distinguished point. The state processes are also centered at this point. The increments of the transition rate functions are assumed to have affine growth. This assumption is relaxed in Corollary 2.1, in which a stronger Foster–Lyapunov criterion is required instead. The technique of proof for this set of results is inspired by [Reference Khasminskii16], which studies stochastic differential equations with rapid Markovian switching. We construct a sequence of Lyapunov functions via Poisson equations associated with the extended generator of the background Markov process. The technique employed for our results is more involved, since a class of Markov processes under weak hypotheses is considered, and the scaling parameter affects the state and background processes at the same time. In the study of ergodicity of a Markov-modulated multiclass $M/M/n + M$ queue under a static priory scheduling policy in [Reference Arapostathis, Das, Pang and Zheng3, Theorem 4], the authors observe an effect of ‘averaged’ Halfin–Whitt regime, and also use a technique similar in spirit to the method in [Reference Khasminskii16]. In this paper, we consider a more general model which includes the one in [Reference Arapostathis, Das, Pang and Zheng3, Theorem 4] as a special case. In Example 3.2, we also show that the result in [Reference Arapostathis, Das, Pang and Zheng3, Theorem 4] holds under some weaker condition, and its proof may be greatly simplified following the approach in Corollary 2.1. In Corollary 2.2 and Remark 2.4, we emphasize that the result in this part can be applied in the study of uniformly exponential ergodicity of Markov-modulated multiclass $M/M/n$ queues with positive safety staffing.

The main result on steady-state approximations is stated in Theorem 2.2. Here, we first construct ‘averaged’ diffusion models, which capture the variabilities of the state process and the underlying Markov process at the same time. In these diffusion models, the variabilities of the state process are asymptotically negligible at a rate $n^{1 - 2\beta}$ when $\alpha < 1$ , while the variabilities of the underlying process are asymptotically negligible at a rate $n^{1 - \alpha}$ when $\alpha > 1$ (see Proposition A.1). The gap between the moments of the steady state of the approximating diffusion models and those of the joint Markov process shrinks at rate of $n^{\alpha/2\wedge 1/2}$ .

The result in Theorem 2.2 extends the results of [Reference Gurvich11] to Markov-modulated birth–death processes. The proofs in [Reference Gurvich11] rely on the gradient estimates of solutions of a sequence of Poisson equations associated with diffusions and a martingale argument. Under a uniformly exponential ergodicity assumption for the diffusion models, the gradient estimates we used for the Poisson equation are the same as those found in [Reference Gurvich11]. However, the martingale argument is difficult to apply in obtaining Theorem 2.2. On the other hand, the proof of [Reference Arapostathis, Das, Pang and Zheng3, Lemma 8] concerning the convergence of mean empirical measures for Markov-modulated multiclass $M/M/n + M$ queues uses a martingale argument, but considers only compactly supported smooth functions. The analogous argument cannot be used in this paper, since we need to consider a class of general birth–death processes and the Lyapunov functions are unbounded. So we develop a new approach by exploring the structural relationship between the generator of the joint Markov process and that of the diffusion models in Lemma 5.1. This is accomplished by matching the second-order derivatives associated with the covariance of the underlying Markov process using the solutions of Poisson equations which involve the difference between the coefficients of the original state process and those of the ‘averaged’ diffusion models. In Lemma 5.2, we also provide some crucial estimates for the residual terms arising from the difference of the two generators.

Stability of switching diffusions has been studied extensively. Exponential stability for nonlinear Markovian switching diffusion processes has been studied in [Reference Mao19], while p-stability and asymptotic stability for regime-switching diffusions have been addressed in [Reference Khasminskii, Zhu and Yin17]. For an underlying Markov process with a countable state space, the stability of regime-switching diffusions has been considered in [Reference Shao and Xi23]. In these studies, the state and background Markov processes are unscaled, and there is no ‘averaged’ system. Under fast regime switching, we observe an ‘averaged’ effect, and study how the ergodic properties of the ‘averaged’ system are related to those of the original system.

1.1. Organization of the paper

The notation used in this paper is summarized in the next subsection. In Section 2, we describe the model of Markov-modulated general birth–death processes. We present the results of exponential ergodicity and steady-state approximations in Sections 2.1 and 2.2, respectively. Section 3 contains some examples from queueing systems. Section 4 is devoted to the proofs of Theorem 2.1 and Corollaries 2.1 and 2.2. The proofs of Theorem 2.2 and Corollary 2.2 are given in Section 5. Proposition A.1 concerning the diffusion limit is given in Appendix A.

1.2. Notation

We let $\mathbb{N}$ and $\mathbb{Z}_+$ denote the set of natural numbers and the set of nonnegative integers, respectively. Let $\mathbb{R}^d$ denote the set of d-dimensional real vectors, for $d\in\mathbb{N}$ . The Euclidean norm and inner product in $\mathbb{R}^d$ are denoted by $\vert{\,\cdot\,}\vert$ and $\langle\,\cdot,\,\cdot\,\rangle$ , respectively. If $a = (a_1,\dots,a_n)$ is an ordered n-tuple, then $\vert{a}\vert := (\!\sum_{i=1}^n a_i^2)^{1/2}$ . For $x\in\mathbb{R}^d$ , $x^{\mathsf{T}}$ denotes the transpose of x. We denote the indicator function of a set $A\subset\mathbb{R}^d$ by $\mathbb{1}_A$ . The minimum (maximum) of $a,b\in\mathbb{R}$ is denoted by $a\wedge b\,(a\vee b)$ , and $a^{\pm} := 0\vee(\!\pm a)$ . We let e denote the vector in $\mathbb{R}^d$ with all entries equal to 1, and $e_i$ the vector in $\mathbb{R}^d$ with the ith entry equal to 1 and all other entries equal to 0. The closure of a set $A\subset\mathbb{R}^d$ is denoted by $\bar{A}$ . The open ball of radius r in $\mathbb{R}^d$ , centered at $x\in\mathbb{R}^d$ , is denoted by $B_r(x)$ .

For a domain $D\subset\mathbb{R}^d$ , the space $C^k(D)$ ( $C^{\infty}(D)$ ) denotes the class of functions whose partial derivatives up to order k (of any order) exist and are continuous, and $C^k_b(D)$ stands for the functions in $C^{k}(D)$ whose partial derivatives up to order k are continuous and bounded. The space $C^{k,1}(D)$ is the class of functions whose partial derivatives up to order k are Lipschitz continuous. We let

\begin{align*}[f]_{2,1;D} := \sup_{x,y\in D, x\neq y}\frac{\bigl\vert{\nabla^2 f(x) - \nabla^2 f(y)}\bigr\vert}{\vert{x - y}\vert}\end{align*}

for a domain $D\subset\mathbb{R}^d$ and $f\in C^{2,1}(D)$ . For a nonnegative function $f\in C(\mathbb{R}^d)$ , we use ${\mathscr{O}}(f)$ to denote the space of functions $g\in C(\mathbb{R}^d)$ such that $\sup_{x\in\mathbb{R}^d} \frac{\vert{g(x)}\vert}{1 + f(x)} < \infty$ . By a slight abuse of notation, we also let ${\mathscr{O}}(f)$ denote a generic member of this space. Given any Polish space $\mathcal{X}$ , we let ${\mathcal{P}}(\mathcal{X})$ denote the space of probability measures on $\mathcal{X}$ , endowed with the Prokhorov metric. For $\mu\in{\mathcal{P}}(\mathcal{X})$ and a Borel measurable map $f: \mathcal{X}\mapsto \mathbb{R}$ , we often use the simplified notation $\mu(f) := \int_{\mathcal{X}} f\,\textrm{d}{\mu}$ .

2. Model and results

Let $Q= [q_{ij}]_{i,j\in{\mathscr{K}}}$ , with ${\mathscr{K}}:=\{1,\dots,k_\circ\}$ , be an irreducible stochastic rate matrix, and let

(2.1) \begin{equation}\pi\,:=\,\{\pi_1,\dots,\pi_{k_\circ}\!\}\end{equation}

denote its (unique) invariant probability distribution. We fix a constant $\alpha>0$ . For each $n\in N$ , let $J^n$ denote the finite-state irreducible continuous-time Markov chain with state space ${\mathscr{K}}$ and transition rate matrix $n^{\alpha}Q$ . In addition, for each $n\in\mathbb{N}$ and $k\in{\mathscr{K}}$ , let ${\mathfrak{X}}^n \subset\mathbb{R}^d$ be a countable set with no accumulation points in $\mathbb{R}^{d}$ , and let $R^n_k=\bigl[r^n_k(x,y)\bigr]_{x,y\in{\mathfrak{X}}^n}$ be a stochastic rate matrix which gives rise to a non-explosive, irreducible, continuous-time Markov chain.

The transition matrices $\{R^n_k\}$ satisfy the following structural assumptions.

Hypothesis 2.1. There exist positive constants $m_0$ , $N_0$ , and $C_0$ such that the following hold for all $x\in{\mathscr{X}}^n$ , $n\in\mathbb{N}$ , and $k\in{\mathscr{K}}$ .

  1. (a) Bounded jumps. It holds that $r^n_k (x,x+z) =0$ for $\vert{z}\vert>m_0$ .

  2. (b) Finitely many jumps. The cardinality of the set

    \begin{equation*}{\mathscr{Z}}^{\,\,n}_k(x) := \{z\in\mathbb{R}^{d}: r^n_k (x,x+z)>0\}\end{equation*}
    does not exceed $N_0$ .
  3. (c) Incremental affine growth. It holds that

    \begin{equation*}\bigl\vert{r^n_k (x,x+z) - r^n_k (x^{\prime},x^{\prime}+z)}\bigr\vert\,\leq\, C_0\bigl(n^{\alpha/2}+\vert{x-x^{\prime}}\vert\bigr).\end{equation*}
  4. (d) There exists some distinguished element $x^n_*\in\mathbb{R}^{d}$ such that

    \begin{equation*}r^n_k (x,x+z) \,\leq\, C_0 (n^{1\vee\alpha/2}+\vert{x-x^n_*}\vert).\end{equation*}

Hypothesis 2.1 is assumed throughout the paper without further mention. We refer the reader to Examples 3.1 to 3.3 for examples of verification of the conditions in Parts (c) and (d).

Remark 2.1. The element $x^n_*\in{\mathfrak{X}}^n$ in Part (d) plays an important role in the analysis. For queueing models, $x^n_*$ may be chosen as the steady state of the ‘average’ fluid; refer to solutions of (2.20) below.

Consider the stochastic rate matrix $S^n$ on ${\mathfrak{X}}^n\times{\mathscr{K}}$ whose elements are defined by

\begin{equation*}s^n\bigl((x,i),(y,j)\bigr) \,:=\, \begin{cases}r^n_i(x,y) \quad &\text{if }\ i=j, \\[4pt] n^{\alpha} q_{ij} \quad &\text{if}\ x=y, \\[4pt] 0 \quad &\text{otherwise},\end{cases}\end{equation*}

for $x,y\in{\mathfrak{X}}^n$ and $i,j\in{\mathscr{K}}$ . This defines a non-explosive, irreducible Markov chain $(X^n,J^n)$ , where $J^n$ is as described in the preceding paragraph.

In order to simplify some algebraic expressions, we often use the notation

\begin{equation*}\tilde{r}^n_k(x,z) = r^n_k(x,x+z).\end{equation*}

Definition 2.1. Let $\beta:=\max\{1/2,1 - \alpha/2\}$ be fixed. With $x^n_*$ as in Hypothesis 2.1(d), we define the scaled process

\begin{equation*}\widehat{X}^n\,:=\, \frac{X^n - {x}^n_{*}}{n^{\beta}}.\end{equation*}

The state space of $\widehat{X}^n$ is given by

\begin{equation*}\widehat{{\mathfrak{X}}}^n \,:=\, \{\hat{x}^n(x) : x\in{\mathfrak{X}}^n\},\end{equation*}

where $\hat{x} = \hat{x}^n(x) := n^{-\beta}(x - x^n_*)$ for $x\in\mathbb{R}^d$ .

Naturally, $(\widehat{X}^n,J^n)$ is a Markov process, and its extended generator is given by

(2.2) \begin{equation}\widehat{\mathscr{\!\!L}}^{\,\,n} f(\hat{x},k) = {\mathscr{L}}_k^{\kern1.6pt n} f(\hat{x},k)+ {\mathcal{Q}}^n f(\hat{x},k),\quad (\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation}

for $f\in C_b(\mathbb{R}^d\times{\mathscr{K}}\,)$ , where

(2.3) \begin{equation}\begin{aligned}{\mathscr{L}}_k^{\kern1.6pt n}f(\hat{x},k) &\,:=\, \sum_{z\in{\mathscr{Z}}^{\,\,n}(x)}\tilde{r}^n_k(n^{\beta}\hat{x} + {x}^n_*,z)\bigl(f(\hat{x} + n^{-\beta}z,k) - f(\hat{x},k)\bigr), \\[4pt] {\mathcal{Q}}^nf(\hat{x},k) &\,:=\, \sum_{\ell\in{\mathscr{K}}}n^{\alpha}q_{k\ell}\bigl(f(\hat{x},\ell) - f(\hat{x},k)\bigr) = \sum_{\ell\in{\mathscr{K}}}n^{\alpha}q_{k\ell}f(\hat{x},\ell).\end{aligned}\end{equation}

One can clearly see that $\,\,\widehat{\mathscr{\!\!L}}^{\,\,n} f$ and ${\mathscr{L}}_k^{\kern1.6pt n}f$ are well defined for $f\in C_b(\mathbb{R}^d)$ , by viewing f as a function on $\mathbb{R}^{d}\times{\mathscr{K}}$ which is constant with respect to its second argument.

Throughout the paper, x and $\hat{x}$ are generic elements of ${\mathfrak{X}}^n$ (or $\mathbb{R}^{d}$ ) and $\widehat{{\mathfrak{X}}}^n$ , respectively.

2.1. Exponential ergodicity

In this subsection, we provide a sufficient condition for the joint process $(\widehat{X}^n,J^n)$ to be exponentially ergodic. We refer the reader to [Reference Meyn and Tweedie20] for the definition of exponential ergodicity and the relevant Foster–Lyapunov criteria. We introduce the following operator, which corresponds to the generator of the ‘averaged’ process.

Definition 2.2. Let

\begin{equation*}\bar{r}^n(x,z) \,:=\, \sum_{k\in{\mathscr{K}}}\pi_k \tilde{r}^n_k(x,z),\end{equation*}

with $\pi_k$ as in (2.1), and

(2.4) \begin{equation}{\mathscr{Z}}^{\,\,n} \,:=\, \cup_{x\in{\mathfrak{X}}^n} \cup_{k\in{\mathscr{K}}} {\mathscr{Z}}^{\,\,n}_k(x).\end{equation}

We define $\overline{\mathscr{L}}^{\,n}: C_b(\mathbb{R}^d\times{\mathscr{K}}\,\kern1.3pt)\mapsto C_b(\mathbb{R}^d\times{\mathscr{K}}\,\kern1.3pt)$ by

(2.5) \begin{equation}\overline{\mathscr{L}}^{\,n} f(\hat{x},k) \,:=\,\sum_{z\in{\mathscr{Z}}^{\,\,n}}\bar{r}^n(n^{\beta}\hat{x} + {x}^n_*,z)\bigl(f(\hat{x} + n^{-\beta}z,k) - f(\hat{x},k)\bigr),\quad (\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation}

for $f\in C_b(\mathbb{R}^d\times{\mathscr{K}}\,)$ .

In the following theorem, we show that if $\overline{\mathscr{L}}^{\,n}$ satisfies a Foster–Lyapunov inequality with a suitable Lyapunov function, then the original joint process $(\widehat{X}^n,J^n)$ is exponentially ergodic. The proof is given in Section 4.

A function $f:\mathbb{R}^d \mapsto \mathbb{R}_+$ is called norm-like if $f(x) \rightarrow\infty$ as $\vert{x}\vert\rightarrow\infty$ ; see, for example, [Reference Meyn and Tweedie20, Section 1.3].

Theorem 2.1. Suppose that there exist a sequence of nonnegative norm-like functions $\{{\mathcal{V}}^n\in C(\mathbb{R}^d): n\in\mathbb{N}\}$ , $n_0\in\mathbb{N}$ , and some positive constants $\varepsilon_0$ , C, $\overline{C}_1$ , $\overline{C}_2$ , not depending on n, such that

(2.6) \begin{equation}\begin{aligned}(1 + \vert{x}\vert)\,\bigl\vert{{\mathcal{V}}^n(x + y) - {\mathcal{V}}^n(x)}\bigr\vert &\,\leq\, C\vert{y}\vert\bigl(1 + {\mathcal{V}}^n(x)\bigr), \\[3pt] \bigl(1 + \vert{x}\vert^2\bigr)\,\bigl\vert{{\mathcal{V}}^n\bigl(x + y + z\bigr) - {\mathcal{V}}^n(x+ y)\mspace{100mu}} &\\ -{\mathcal{V}}^n(x+ z) + {\mathcal{V}}^n(x)\bigr\vert &\,\leq\, C\vert{y}\vert\vert{z}\vert\bigl(1 + {\mathcal{V}}^n(x)\bigr),\end{aligned}\end{equation}

for any $y,z\in B_0(\varepsilon_0)\setminus\{0\}$ , $x\in\mathbb{R}^d$ , and $n\in\mathbb{N}$ , and

(2.7) \begin{equation}\overline{\mathscr{L}}^{\,n} {\mathcal{V}}^n(\hat{x}) \,\leq\, \overline{C}_1 - \overline{C}_2{\mathcal{V}}^n(\hat{x})\qquad \forall\,\hat{x}\in{\widehat{{\mathfrak{X}}}^n},\ \forall\, n > n_0.\end{equation}

Then there exist functions $\widehat{{\mathcal{V}}}^n\in C(\mathbb{R}^d\times{\mathscr{K}}\,)$ and positive constants $\widehat{C}_1$ , $\widehat{C}_2$ , and $n_1\in\mathbb{N}$ such that, for all $n\geq n_1$ , we have

(2.8) \begin{equation}\frac{1}{2}\bigl({\mathcal{V}}^n(\hat{x}) - 1\bigr) \,\leq\, \widehat{{\mathcal{V}}}^n(\hat{x},k) \,\leq\,\frac{3}{2}{\mathcal{V}}^n(\hat{x}) +\frac{1}{2}\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation}

and

(2.9) \begin{equation}\widehat{\mathscr{\!\!L}}^{\,\,n} \widehat{{\mathcal{V}}}^n(\hat{x},k) \,\leq\, \widehat{C}_1 - \widehat{C}_2\widehat{{\mathcal{V}}}^n(\hat{x},k) \qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\ \forall\, n>n_1.\end{equation}

As a consequence, $(\widehat{X}^n,J^n)$ is exponentially ergodic for all $n> n_1$ , and its invariant probability distributions are tight.

Remark 2.2. It follows from the proof of Theorem 2.1 that $\widehat{C}_2$ can be selected arbitrarily close to $\overline{C}_2$ , so the rates of convergence of the ‘averaged’ system and the Markov-modulated one become asymptotically close.

Remark 2.3. A sufficient condition for a function ${\mathcal{V}}^n\in C^{2,1}(\mathbb{R}^{d})$ to satisfy (2.6) is

(2.10) \begin{equation}\begin{aligned}\vert{\nabla {\mathcal{V}}^n(x)}\vert &\,\leq\, c\frac{1+{\mathcal{V}}^n(x)}{1+\vert{x}\vert}\quad\text{and}\\[5pt] \vert{\nabla^2 {\mathcal{V}}^n(x)}\vert +\bigl[{\mathcal{V}}^n\bigr]_{2,1;B_{\epsilon}(x)}&\,\leq\, c\frac{1+{\mathcal{V}}^n(x)}{1+\vert{x}\vert^2}\quad\forall\,x\in\mathbb{R}^{d},\end{aligned}\end{equation}

for some fixed positive constants $\epsilon$ and c.

In the next corollary, we relax the incremental growth hypothesis in Hypothesis 2.1(c). The proof is contained in Section 4. In Example 3.2, we show that this result can be applied in the study of exponential ergodicity for Markov-modulated $M/M/n +M$ queues.

We replace Hypothesis 2.1(c) by the following weaker assumption.

Assumption 2.1 Suppose that Parts (a), (b), and (d) of Hypothesis 2.1 are satisfied, and $\tilde r^n_k$ can be decomposed into

\begin{equation*}\tilde{r}^n_k(x,z) = \phi^n_k(x,z) + \psi^n_k(x,z),\quad x\in{\mathfrak{X}}^n, \quad z\in{\mathscr{Z}}^{\,\,n}_k(x),\end{equation*}

where $\phi^n_k(x,z)$ and $\psi^n_k(x,z)$ , $k\in{\mathscr{K}}$ , are locally bounded functions on ${\mathfrak{X}}^n\times{\mathscr{Z}}^{\,\,n}$ . In addition, using without loss of generality the same constant, there exist $\delta_1,\delta_2 \in[0,1]$ such that

(2.11) \begin{equation}\vert{\psi^n_k(x,z) - \psi^n_k(y,z)}\vert\,\leq\, C_0\bigl(n^{\alpha/2} + \vert{x - y}\vert^{\delta_1}\bigr) \qquad \forall\,k\in{\mathscr{K}}, \ \forall\,x,y\in{\mathfrak{X}}^n, \ \forall\,z\in{\mathscr{Z}}^{\,\,n},\end{equation}

and

(2.12) \begin{equation}\vert{\psi^n_k(x,z)}\vert \,\leq\, C_0\bigl(n^{1\vee\alpha/2}+ \vert{x - x^n_*}\vert^{\delta_2}\bigr) \qquad \forall\,k\in{\mathscr{K}},\ \forall\,(x,z)\in{\mathfrak{X}}^n\times{\mathscr{Z}}^{\,\,n},\end{equation}

with $x^n_*\in\mathbb{R}^d$ as in Hypothesis 2.1(d), and for $n\in\mathbb{N}$ .

Corollary 2.1. Grant Assumption 2.1. Let ${\mathcal{G}}^n_k: C_b(\mathbb{R}^d\times{\mathscr{K}}\,)\mapsto C_b(\mathbb{R}^d\times{\mathscr{K}}\,)$ be defined by

(2.13) \begin{equation}{\mathcal{G}}^n_k f(\hat{x},k) \,:=\,\sum_{z\in{\mathscr{Z}}^{\,\,n}}\bigl(\phi^n_k(n^{\beta}\hat{x} + {x}^n_*,z)+ \bar{\psi}^n(n^{\beta}\hat{x} + {x}^n_*,z)\bigr)\bigl(f(\hat{x} + n^{-\beta}z,k) - f(\hat{x},k)\bigr)\end{equation}

for $(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}$ and $f\in C_b(\mathbb{R}^d\times{\mathscr{K}}\,)$ , and with $x^n_*$ as in Assumption 2.1, where $\bar{\psi}^n(x,z) := \sum_{k\in{\mathscr{K}}}\pi_k\psi^n_k(x,z)$ . Suppose that (2.6) holds with the second inequality replaced by

\begin{equation*}\bigl(1 + \vert{x}\vert^{1+\delta_2}\bigr)\,\bigl\vert{{\mathcal{V}}^n\bigl(x + y + z\bigr) - {\mathcal{V}}^n(x+ y) -{\mathcal{V}}^n(x+ z) + {\mathcal{V}}^n(x)}\bigr\vert \,\leq\, C\vert{y}\vert\vert{z}\vert\bigl(1 + {\mathcal{V}}^n(x)\bigr),\end{equation*}

where $\delta_2$ is as in Assumption 2.1, and there exist $n_2\in\mathbb{N}$ and some positive constants $C_1$ and $C_2$ such that

(2.14) \begin{equation}{\mathcal{G}}^n_k {\mathcal{V}}^n(\hat{x}) \,\leq\, C_1 - C_2{\mathcal{V}}^n(\hat{x})\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}, \ \forall\, n > n_2.\end{equation}

Then the results in (2.8) and (2.9) hold.

In the following corollary, we show that under some stronger assumptions on the transition rate functions and the scaling parameters, (2.6) can be weakened. The proof is given in Section 4.

Corollary 2.2. Grant parts (a) and (b) of Hypothesis 2.1, and suppose that $r^n_k$ satisfies

(2.15) \begin{equation}\bigl\vert{r^n_k (x,x+z) - r^n_k (x^{\prime},x^{\prime}+z)}\bigr\vert\,\leq\, C_0\bigl(1+\vert{x-x^{\prime}}\vert\wedge n\bigr)\end{equation}

and

(2.16) \begin{equation}r^n_k (x^n_*,x^n_*+z)\,\leq\, C_0 n.\end{equation}

If in the assumptions of Theorem 2.1 we replace (2.6) by

(2.17) \begin{equation}\begin{aligned}\bigl\vert{{\mathcal{V}}^n(x + y) - {\mathcal{V}}^n(x)}\bigr\vert &\,\leq\, C\vert{y}\vert\bigl(1 + {\mathcal{V}}^n(x)\bigr), \\\bigl\vert{{\mathcal{V}}^n\bigl(x + y + z\bigr) - {\mathcal{V}}^n(x+ y) -{\mathcal{V}}^n(x+ z) + {\mathcal{V}}^n(x)}\bigr\vert &\,\leq\, C\vert{y}\vert\vert{z}\vert\bigl(1 + {\mathcal{V}}^n(x)\bigr),\end{aligned}\end{equation}

then, provided $\beta$ and $\alpha$ satisfy $2\beta+\alpha>2$ , the conclusions of the theorem still hold.

Note that (2.17) is satisfied for exponential functions.

Remark 2.4. The transition rates of multiclass $M/M/n$ queues, that is, the model in Example 3.2 with no abandonment ( $\gamma_i(k)\equiv0$ ), satisfy (2.15) and (2.16). Uniform exponential ergodicity of this model (with spare capacity, or equivalently, positive safety staffing) is established in [Reference Arapostathis, Hmedi and Pang4] using exponential Lyapunov functions. Thus, we may use exponential Lyapunov functions in (2.7), and take advantage of the results in [Reference Arapostathis, Hmedi and Pang4] to establish exponential ergodicity of Markov-modulated multiclass $M/M/n$ queues with positive safety staffing using the Lyapunov functions in [Reference Arapostathis, Hmedi and Pang4]. We leave it to the reader to verify that for $\alpha\geq1$ , we can in fact establish uniform exponential ergodicity over all work-conserving scheduling policies. For $\alpha<1$ the discontinuities allowed in the policies need to be restricted.

Extending this to the classes of multiclass multi-pool models studied in [Reference Hmedi, Arapostathis and Pang12] is also possible.

2.2. Steady-state approximations

Here, we use a function $\xi^n_z(x,k)$ for $(x,z)\in\mathbb{R}^{d}\times\mathbb{R}^{d}$ and $k\in{\mathscr{K}}$ which interpolates the transition rates in the sense that

\begin{equation*}\xi^n_{z}(x,k) = r^n_k(x,x+z) \quad \text{if }\ x,x+z\in{\mathfrak{X}}^n.\end{equation*}

Recall the definition of ${\mathscr{Z}}^{\,\,n}$ in (2.4). It is clear that for $z\notin{\mathscr{Z}}^{\,\,n}$ we may let $\xi^n_z\equiv0$ . Thus

\begin{equation*}{\mathscr{Z}}^{\,\,n} = \bigl\{z\in\mathbb{R}^{d}: \exists\, x, k\ \text{such}\ \text{that}\ \xi^n_{z}(x,k)>0\}.\end{equation*}

This of course also implies that

(2.18) \begin{equation}\xi^n_{z}(x,k) = 0 \quad\text{if}\ \vert{z}\vert> m_0\end{equation}

by Hypothesis 2.1(a).

We let ${\mathscr{I}}:=\{1,\dotsc,d\}$ , and define

(2.19) \begin{equation}\begin{aligned}\Xi^n(x,k) &\,:=\, \sum_{z\in{\mathscr{Z}}^{\,\,n}} z\,\xi^n_{z}(x,k),\\[5pt] \Gamma^n_{ij}(x,k) &\,:=\, \sum_{z\in{\mathscr{Z}}^{\,\,n}}z_iz_j\xi^n_z(x,k),\quad i,j\in{\mathscr{I}},\end{aligned}\end{equation}

for $(x,k)\in \mathbb{R}^d\times{\mathscr{K}}$ .

We impose the following structural assumptions on the function $\xi^n$ .

Assumption 2.2 The following hold.

  1. (i) The cardinality of the set $\{z\in\mathbb{R}^{d}: \xi^n_{z}(x,k)>0\}$ does not exceed $\widetilde{N}_0$ .

  2. (ii) For each $n\in\mathbb{N}$ , there exists $x^n_*\in\mathbb{R}^{d}$ satisfying

    (2.20) \begin{equation}\sum_{k\in{\mathscr{K}}}\pi_k\Xi^n({x}^n_*,k) = 0.\end{equation}
  3. (iii) The function $\xi^n_z$ is uniformly Lipschitz continuous in its first argument; that is, there exists some positive constant $\widetilde{C}$ such that

    (2.21) \begin{equation}\vert{\xi^n_z(x,k) - \xi^n_z(y,k)}\vert\,\leq\, \widetilde{C}\vert{x - y}\vert \qquad \forall\,k\in{\mathscr{K}},\ \forall\,x,y\in\mathbb{R}^d, \ \forall\,z\in{\mathscr{Z}}^{\,\,n},\end{equation}
    for all $n\in\mathbb{N}$ . In addition, using without loss of generality the same constant, we assume that
    (2.22) \begin{equation}\max_{z\in\mathbb{R}^{d}}\,\xi^n_z(x^n_*, k)\,\leq\,\widetilde{C} n\qquad \forall\, k\in{\mathscr{K}},\ \forall\,n\in\mathbb{N}.\end{equation}
  4. (iv) The matrix $\Gamma^n(x^n_*,k)$ is positive definite, and

    (2.23) \begin{equation}\frac{1}{n}\, \Gamma^n(x^n_*, k) \,\xrightarrow[n\to\infty]{}\, \bar{\Gamma}(k),\end{equation}
    where $\bar{\Gamma}(k)$ is also a positive definite $d\times d$ matrix, for all $k\in{\mathscr{K}}$ .

We note here that the nondegeneracy hypothesis in Assumption 2.2(iv) is used in Lemma 5.3 to derive gradient estimates of the solution of a Poisson equation.

Remark 2.5. Equation (2.21) is of course much stronger than Hypothesis 2.1(c). This is needed for the results in this section which rely on certain Schauder estimates for solutions of the Poisson equation for the generator of an approximating diffusion equation.

Let $\{A^n_z : z\in {\mathscr{Z}}^{\,\,n}\}$ be a family of independent unit-rate Poisson processes, independent of $J^n$ , and $\tilde{A}^n_{z}(t) := A^n_{z}(t) - t$ . Then the d-dimensional process $X^n(t)$ is governed by the equation

\begin{equation}\begin{aligned}X^n(t) & = X^n(0) +\sum_{z\in{\mathscr{Z}}^{\,\,n}} z\, A^n_{z}\biggl(\int_0^t \xi^n_{z}\bigl(X^n(s),J^n(s)\bigr)\,\textrm{d}{s}\biggr)\nonumber\\ & = X^n(0) + M^n(t) +\int_0^t \Xi^n\bigl(X^n(s),J^n(s)\bigr)\,\textrm{d}{s} ,\end{aligned}\end{equation}

where

\begin{equation*}M^n(t)\,:=\,\sum_{z\in{\mathscr{Z}}^{\,\,n}}z \,\tilde{A}^n_{z}\biggl(\int_0^t\xi^n_{z}\bigl(X^n(s),J^n(s)\bigr)\,\textrm{d}{s}\biggr).\end{equation*}

Note that $M^n(t)$ is a local martingale with respect to the filtration

\begin{equation*}\begin{aligned}\mathcal{F}^n_t &\,:=\, \sigma\biggl\{X^n(0),J^n(s),\tilde{A}^n_{z}\biggl(\int_0^t\xi^n_{z}\bigl(X^n(s),J^n(s)\bigr)\,\textrm{d}{s}\biggr), \\[5pt] &\mspace{120mu} \int_0^t\xi^n_{z}\bigl(X^n(s),J^n(s)\bigr)\,\textrm{d}{s}: z\in{\mathscr{Z}}^{\,\,n}, s\leq t\biggr\}.\end{aligned}\end{equation*}

The locally predictable quadratic variation of $M^n$ satisfies

\begin{equation*}\langle M^n \rangle (t) = \int_0^t \Gamma^n\bigl(X^n(s),J^n(s)\bigr)\,\textrm{d}{s},\quad t\geq0,\end{equation*}

where the function $\Gamma^n = [\Gamma^n_{ij}]: \mathbb{R}^d\times{\mathscr{K}} \mapsto \mathbb{R}^{d\times d}$ is given in (2.19).

By (2.21), it is evident that given $x^n(0) \in\mathbb{R}^d$ , there exists a unique solution $x^n(t)$ satisfying

\begin{equation*}x^n(t) = x^n(0) + \sum_{k\in{\mathscr{K}}}\pi_k\int_0^t\Xi^n(x^n(s),k)\,\textrm{d}{s} .\end{equation*}

We refer to this as the nth ‘averaged’ fluid model.

In this section, the scaled process is defined as in Definition 2.1, with the exception that $x^n_*\in\mathbb{R}^{d}$ is as specified in Assumption 2.2. Note that in the extended generator in (2.2) and (2.3) we may replace $\tilde{r}^n_k(n^{\beta}\hat{x} + {x}^n_*,z)$ by $\xi^n_z(n^{\beta}\hat{x} + {x}^n_*, k)$ . It is evident from (2.24) that $\widehat{X}^n$ satisfies

(2.24) \begin{equation}\widehat{X}^n(t) = \widehat{X}^n(0) + \widehat{M}^n(t)+ \int_0^t\widehat{\Xi}^n\bigl(\widehat{X}^n(s),J^n(s)\bigr)\,\textrm{d}{s},\end{equation}

where

(2.25) \begin{equation}\widehat{M}^n\,:=\, \frac{M^n}{n^{\beta}},\qquad \text{and} \quad\widehat{\Xi}^n(\hat{x},k)\,:=\,\frac{\Xi^n(n^{\beta}\hat{x} + {x}^n_{*},k)}{n^{\beta}},\qquad (\hat{x},k)\in\mathbb{R}^d\times{\mathscr{K}}.\end{equation}

The locally predictable quadratic variation of $\widehat{M}^n$ is given by

\begin{equation*}\langle \widehat{M}^n \rangle (t) = \int_0^{t}\bar{\Gamma}^n\bigl(\widehat{X}^n(s),J^n(s)\bigr)\,\textrm{d}{s},\qquad t\geq0,\end{equation*}

with

(2.26) \begin{equation}\bar{\Gamma}^n(\hat{x},k) \,:=\, \frac{1}{n^{2\beta}}\,\Gamma^n(n^{\beta}\hat{x} + {x}^n_*,k),\qquad (\hat{x},k)\in\mathbb{R}^d\times{\mathscr{K}}.\end{equation}

We next introduce a sequence of processes that approximate $\widehat{X}^n$ . Let $\widehat{Y}^n$ be the strong solution to the Itô d-dimensional stochastic differential equation

(2.27) \begin{equation}\textrm{d} \widehat{Y}^n(t) = \bar{b}^n\bigl(\widehat{Y}^n(t)\bigr)\,\textrm{d}{t}+ \sigma^n \textrm{d} W(t),\end{equation}

with $\widehat{Y}^n(0) = y_0$ , where W(t) is a d-dimensional standard Brownian motion,

\begin{equation*}\bar{b}^n_i(\hat{y}):= \sum_{k\in{\mathscr{K}}}\pi_k\widehat{\Xi}^n_i(\hat{y},k),\qquad \,\hat{y}\in\mathbb{R}^d, \ i\in{\mathscr{I}},\end{equation*}

with $\widehat{\Xi}^n$ defined in (2.25). The diffusion matrix $\sigma^n$ is characterized as follows. Let

(2.28) \begin{equation}\Upsilon \,:=\, (\Pi - Q)^{-1} - \Pi\end{equation}

denote the deviation matrix corresponding to the transition rate matrix Q [Reference Coolen-Schrijner and van Doorn7]. Let $\Theta^n = [\theta^n_{ij}]$ be defined by

(2.29) \begin{equation}\theta^n_{ij}\,:=\, 2\sum_{\ell\in{\mathscr{K}}}\sum_{k\in{\mathscr{K}}}\frac{\Xi^n_i({x}^n_*,k)\Xi^n_j({x}^n_*,\ell)}{n^{\alpha + 2\beta}}\pi_k\Upsilon_{k\ell} ,\qquad i,j\in{\mathscr{I}},\end{equation}

and

\begin{equation*}\bar{a}^n(x) = [\bar{a}^n_{ij}](x)\,:=\, \sum_{k\in{\mathscr{K}}}\pi_k\bar{\Gamma}^n(x,k), \qquad x\in\mathbb{R}^d.\end{equation*}

Then, by Assumption 2.2(iv), and using the spectral decomposition, $\sigma^n$ satisfies

(2.30) \begin{equation}\Sigma^{n} := (\sigma^n)^{\mathsf{T}}\sigma^n = \bar{a}^n(0) + \Theta^n.\end{equation}

The generator of $\widehat{Y}^n$ , denoted by ${\mathscr{A}}^n$ , is given by

(2.31) \begin{equation}{\mathscr{A}}^n f(x) = \sum_{i\in{\mathscr{I}}}\bar{b}^n_i(x)\,\partial_i f(x) +\frac{1}{2}\sum_{i,j\in{\mathscr{I}}}\Sigma^{n}_{ij}\,\partial_{ij}f(x),\qquad f\in C^2(\mathbb{R}^d).\end{equation}

We borrow the following definitions from [Reference Gurvich11]. We say that a function $f\in C^2(\mathbb{R}^d)$ is sub-exponential if $f \geq 1$ and there exists some positive constant c such that

\begin{equation*}\vert{\nabla f(x)}\vert + \bigl\vert{\nabla^2 f(x)}\bigr\vert \,\leq\, c\,\textrm{e}^{c\vert{x}\vert} \qquad\forall\,x\in\mathbb{R}^d\end{equation*}

and

\begin{equation*}\sup_{\{z: \vert{z}\vert\leq1\}}\frac{f(x+z)}{f(x)} \,\leq\, c \qquad\forall\,x\in\mathbb{R}^d.\end{equation*}

We also let ${\mathscr{B}}_x$ denote the open ball around $x\in\mathbb{R}^{d}$ of radius $(1+\vert{x}\vert)^{-1}$ , and define

\begin{equation*}\Vert{f}\Vert_{C^{0,1}({\mathscr{B}}_x)} \,:=\, \sup_{y\in{\mathscr{B}}_x}\vert{f(x)}\vert+ \sup_{y,z\in{\mathscr{B}}_x}\frac{\vert{f(y)-f(z)}\vert}{\vert{y-z}\vert},\qquad f\in C^{0,1}(\mathbb{R}^{d}).\end{equation*}

The following assumption concerning the ergodic properties of $\widehat{Y}^n$ plays a crucial role in the proofs for steady-state approximations.

Assumption 2.3 There exist a sub-exponential norm-like function ${\mathscr{V}}\in C^2(\mathbb{R}^d)$ , a positive constant $\kappa$ , and an open ball ${\mathscr{B}}$ such that

\begin{equation*}{\mathscr{A}}^n {\mathscr{V}}(x) \,\leq\, \mathbb{1}_{{\mathscr{B}}}(x) - \kappa{\mathscr{V}}(x) \qquad\forall\, x\in\mathbb{R}^d,\ \forall\,n\in\mathbb{N}.\end{equation*}

We continue with the main result of this section. Its proof is given in Section 5. Let $\nu^n\in{\mathcal{P}}(\mathbb{R}^d)$ denote the steady-state distribution of $\widehat{Y}^n$ .

Theorem 2.2. Grant Assumptions 2.2 and 2.3. Assume that $(\widehat{X}^n,J^n)$ is ergodic, and its steady-state distribution $\pi^n\in{\mathcal{P}}(\mathbb{R}^d\times{\mathscr{K}}\,)$ satisfies

(2.32) \begin{equation}\limsup_{n\rightarrow\infty}\int_{\mathbb{R}^d\times{\mathscr{K}}}{\mathscr{V}}(\hat{x})(1 + \vert{\hat{x}}\vert)^5\pi^n(\textrm{d}{\hat{x}},\textrm{d}{k})\,<\,\infty.\end{equation}

Then, for any $f : \mathbb{R}^d \mapsto \mathbb{R}$ such that $\Vert{f}\Vert_{C^{0,1}({\mathscr{B}}_x)}\leq {\mathscr{V}}(x)$ , and any $\alpha >0$ , we have

(2.33) \begin{equation}\vert{\pi^n(f) - \nu^n(f)}\vert = {\mathscr{O}}\biggl(\frac{1}{n^{\alpha/2\wedge 1/2 }}\biggr).\end{equation}

Theorem 2.2 concerns the gap between the moments of the marginal distribution of the steady-state $\hat{X}^n$ and those of $\nu^n$ . The order of the function in (2.32) is determined by the estimates in Lemma 5.2 and the gradient estimates of the solutions to the Poisson equation in Lemma 5.3. In the following corollary, we provide a sufficient condition for (2.32). We give its proof in Section 5. In Section 3, we show that this sufficient condition holds in many examples.

Corollary 2.3. Grant Assumption 2.2. Let ${\mathscr{V}}$ and $\widetilde{{\mathscr{V}}}$ be two sub-exponential functions in $C^2(\mathbb{R}^d)$ satisfying Assumption 2.3, such that

(2.34) \begin{equation}{\mathscr{V}}(x)(1 + \vert{x}\vert^5) \,\leq\, \widetilde{{\mathscr{V}}}(x)\end{equation}

and

(2.35) \begin{equation}(1 + \vert{x}\vert)\bigl( \vert{\nabla{\widetilde{{\mathscr{V}}}(x)}}\vert+ \bigl\vert{\nabla^2{\widetilde{{\mathscr{V}}}(x)}}\bigr\vert \bigr)+ (1 + \vert{x}\vert^2)\bigl[\widetilde{{\mathscr{V}}}\bigr]_{2,1;B_{m_0/n^{\beta}}(x)}\,\leq\, C \widetilde{{\mathscr{V}}}(x)\end{equation}

for some positive constant C and any $x\in\mathbb{R}^d$ , with $m_0$ as in (2.18). Then (2.32) holds for ${\mathscr{V}}$ . As a consequence, (2.33) holds.

3. Examples

In this section, we demonstrate how the results of Section 2 can be applied through examples.

Example 3.1. (Markov-modulated $M/M/\infty$ queue.) We consider a process given by

\begin{equation*}{X}^n(t) \,:=\, {X}^n(0)+ A^{n}_1\biggl(\int_0^{t}n\lambda\bigl(J^n(s)\bigr)\,\textrm{d}{s}\biggr)- A^{n}_{-1}\biggl(\int_0^{t}\mu\bigl(J^n(s)\bigr) X^n(s)\,\textrm{d}{s}\biggr),\end{equation*}

where $ A^{n}_{-1}$ and $A^{n}_1$ are mutually independent unit-rate Poisson processes, independent of $J^n$ , for $n\in\mathbb{N}$ . We assume that $\lambda(k) > 0$ and $\mu(k) > 0$ , for $k\in{\mathscr{K}}$ . We let

(3.1) \begin{equation}{x}^n_{*} = n\frac{\sum_{k\in{\mathscr{K}}}\pi_k\lambda(k)}{\sum_{k\in{\mathscr{K}}}\pi_k\mu(k)}.\end{equation}

Recall that $\widehat{X}^n = n^{-\beta}(X^n - x^n_*)$ , and then $\widehat{{\mathfrak{X}}}^n = \{\hat{x}^n(x) : x\in\mathbb{Z}_+ \}$ . It is evident that $\lambda(k)$ and $\mu(k)x$ satisfy Hypothesis 2.1(c)–(d). Let $\bar{\lambda} := \sum_{k\in{\mathscr{K}}}\pi_k\lambda(k)$ and $\bar{\mu} := \sum_{k\in{\mathscr{K}}}\pi_k\mu(k)$ . By Definition 2.2, we obtain

(3.2) \begin{equation}\overline{\mathscr{L}}^{\,n} f(\hat{x}) = n\bar{\lambda}\,\bigl(f(\hat{x} + n^{-\beta}) - f(\hat{x})\bigr)+ \bar{\mu}\,(n^{\beta}\hat{x} + x^n_*)\bigl(f(\hat{x} - n^{-\beta}) - f(\hat{x})\bigr)\end{equation}

for all $\hat{x}\in\widehat{{\mathfrak{X}}}^n$ . Let ${\mathcal{V}}(x) = \vert{x}\vert^m$ , for $x\in\mathbb{R}$ , with $m \geq 2$ an even integer. It is clear that

(3.3) \begin{equation}\vert{\hat{x} \pm n^{-\beta}}\vert^m - \vert{\hat{x}}\vert^m = \pm n^{-\beta}m(\hat{x})^{m-1}+ {\mathscr{O}}(n^{-2\beta}){\mathscr{O}}(\vert{\hat{x}}\vert^{m-2}).\end{equation}

Thus we obtain from (3.1) and (3.2) that

\begin{equation*}\begin{aligned}\overline{\mathscr{L}}^{\,n} {\mathcal{V}}(\hat{x}) & = n\bar{\lambda}\bigl(\vert{\hat{x}+n^{-\beta}}\vert^m - \vert{\hat{x}}\vert^m - n^{-\beta}m\vert{\hat{x}}\vert^{m-1}\bigr)+ \bar{\mu} n^{\beta} \hat{x}\bigl(\vert{\hat{x}-n^{-\beta}}\vert^m - \vert{\hat{x}}\vert^m\bigr) \\ &\qquad + \bar{\mu} {x}^n_{*}\bigl(\vert{\hat{x}-n^{-\beta}}\vert^m - \vert{\hat{x}}\vert^m+ mn^{-\beta}\vert{\hat{x}}\vert^{m-1}\bigr) \\ & = \bar{\lambda}{\mathscr{O}}(n^{1-2\beta}){\mathscr{O}}(\vert{\hat{x}}\vert^{m-2})+ \bar{\mu}\bigl(-\vert{\hat{x}}\vert^m + {\mathscr{O}}(n^{-\beta}){\mathscr{O}}(\vert{\hat{x}}\vert^{m-1}) \\ &\mspace{250mu}+ {\mathscr{O}}(n^{1-2\beta}){\mathscr{O}}(\vert{\hat{x}}\vert^{m-2}) \bigr) \\ & \leq C_1 - C_2{\mathcal{V}}(\hat{x}) \qquad \forall\,\hat{x}\in\widehat{{\mathfrak{X}}}^n,\end{aligned}\end{equation*}

for some positive constants $C_1$ and $C_2$ , where in the second equality we use (3.3), and in the last line we apply Young’s inequality. It is straightforward to verify that ${\mathcal{V}}(x)$ satisfies (2.10). Therefore, the assumptions in Theorem 2.1 hold, and $(\widehat{X}^n,J^n)$ is exponentially ergodic for all large enough n.

Next we verify the assumptions in Corollary 2.3. The equation in (2.20) becomes

(3.4) \begin{equation}\sum_{k\in{\mathscr{K}}}\pi_k\Xi^n(x^n_*,k) = \sum_{k\in{\mathscr{K}}}\pi_k n\lambda(k)- \sum_{k\in{\mathscr{K}}}\pi_k\mu(k)x^n_* = 0.\end{equation}

Note that $x^n_*$ in (3.1) is the unique solution to (3.4). Recall the representation of $\widehat{Y}^n$ in (2.27). In this example, it follows by (3.4) that

\begin{equation*}\bar{b}^n(x) = n^{-\beta}\bar{\mu}x^n_* - n^{-\beta}\bar{\mu} (n^{\beta}x + x^n_*) = -\bar{\mu}x \qquad \forall\,x\in\mathbb{R},\end{equation*}

and

\begin{equation*}\bar{a}^n(0) = n^{-2\beta}(n\bar{\lambda} + \bar{\mu}x^n_*) = n^{1-2\beta}2\bar{\lambda}.\end{equation*}

Let ${\mathscr{V}}(x) = \kappa + \vert{x}\vert^m$ , with $\kappa \geq 1$ , for some integer $m\geq 2$ . We choose some $\tilde{\kappa}\geq1$ such that

\begin{equation*}\widetilde{{\mathscr{V}}}(x) \,:=\, \tilde{\kappa}\bigl(1 + \vert{x}\vert^{5+m}\bigr)\,\geq\, {\mathscr{V}}(x)(1 + \vert{x}\vert^5)\qquad\forall\,x\in\mathbb{R}.\end{equation*}

Then Assumptions 2.2 and 2.3 are satisfied. Indeed, by the discussion following Theorem 3.1 of [Reference Gurvich11], if $\widetilde{{\mathscr{V}}}\in C^3(\mathbb{R}^d)$ in Corollary 2.3, we may replace (2.35) by

(3.5) \begin{equation}(1 + \vert{x}\vert)\bigl(\vert{\nabla{\widetilde{{\mathscr{V}}}(x)}}\vert+ \bigl\vert{\nabla^2{\widetilde{{\mathscr{V}}}(x)}}\bigr\vert \bigr)+ (1 + \vert{x}\vert^2)\bigl\vert{\nabla^3{\widetilde{{\mathscr{V}}}(x)}}\bigr\vert\,\leq\, C \widetilde{{\mathscr{V}}}(x)\end{equation}

for some positive constant C and any $x\in\mathbb{R}^d$ , where

\begin{equation*}\nabla^3 := \frac{\partial^3}{\partial x_1^{\eta_1}\cdots\partial x_d^{\eta_d}}\end{equation*}

with a multi-index $(\eta_1,\dots,\eta_d)$ satisfying $\sum_{i=1}^d\eta_i = 3$ . Then it is straightforward to check that $\widetilde{{\mathscr{V}}}$ chosen above satisfies (3.5). Thus, the result in Corollary 2.3 follows.

The following example concerns Markov-modulated multiclass $M/M/n + M$ queues. Exponential ergodicity for these queues under a static priority scheduling policy has been studied in [Reference Arapostathis, Das, Pang and Zheng3, Theorem 4], which treats a special case of the model considered in this paper. Here we show that by using the result in Corollary 2.1, the proof of [Reference Arapostathis, Das, Pang and Zheng3, Theorem 4] is greatly simplified. We also extend the results in [Reference Arapostathis, Das, Pang and Zheng3, Theorem 4 and Lemma 3] to include a larger class of scheduling policies such that the Markov-modulated queues have exponential ergodicity.

Example 3.2. (Markov-modulated multiclass $M/M/n + M$ queues.) We consider a d-dimensional birth–death process $\{X^n(t): t\geq0\}$ , with state space $\mathbb{Z}^d_+$ , given by

\begin{equation*}\begin{aligned}{X}^n_i(t) &\,:=\, {X}^n_i(0)+ A^{n}_{e_i}\biggl(\int_0^{t}n\lambda_i\bigl(J^n(s)\bigr)\,\textrm{d}{s}\biggr) \\[4pt] &\qquad- A^{n}_{-e_i}\biggl(\int_0^{t}\Bigl(\mu_i\bigl(J^n(s)\bigr)z^n_i(X^n(s))+ \gamma_i\bigl(J^n(s)\bigr)\bigl(X^n_i(s) - z^n_i(X^n(s))\bigr)\Bigr)\,\textrm{d}{s}\biggr)\end{aligned}\end{equation*}

for $i\in{\mathscr{I}}:=\{1,\dots,d\}$ , where $\{A^{n}_{e_i},A^{n}_{-e_i}: i\in{\mathscr{I}}\}$ are mutually independent unit-rate Poisson processes, independent of $J^n$ , and $z^n$ is the static priority policy defined by

\begin{equation*}z^n_i(x) \,:=\, x_i\wedge\biggl(n - \sum_{j=1}^{i-1}x_j\biggr)^+\qquad \forall\,i\in{\mathscr{I}}.\end{equation*}

We assume that $\{\lambda_i(k),\mu_i(k),\gamma_i(k): i\in{\mathscr{I}},k\in{\mathscr{K}}\}$ are strictly positive, and the system is critically loaded, that is, $\sum_{i\in{\mathscr{I}}}\rho_i = 1$ with $\rho_i := \bar{\lambda}_i/\bar{\mu}_i$ . Equation (2.20) becomes

\begin{equation*}\sum_{k\in{\mathscr{K}}}\pi_k \Xi^n_i({x}^n_*,k) = n\bar{\lambda}_i - \bar{\mu}_iz^n_i(x^n_*)- \bar{\gamma}_i(x^n_{*,i} - z^n_i(x^n_*)) = 0\qquad \forall\,i\in{\mathscr{I}},\end{equation*}

which has a unique solution $x^n_* = n\rho$ with $\rho = (\rho_1,\dots,\rho_d)$ .

We first establish exponential ergodicity and verify Assumption 2.1. Let

\begin{equation*}\psi^n_{e_i}(x,k) = n \lambda_i(k),\qquad \psi^n_{-e_i}(x,k) = n \rho_i \mu_i(k),\end{equation*}

and

\begin{equation*}\phi^n_{-e_i}(x,k) = \mu_i(k)(z^n_i(x) - n\rho_i)+ \gamma_i(k)(x_i - z^n_i(x))\end{equation*}

for $i\in{\mathscr{I}}$ and $(x,k)\in\mathbb{R}^d\times{\mathscr{K}}$ . Then $\bar{\psi}^n_{e_i}(x) = n\bar{\lambda}_i$ and $\bar{\psi}^n_{-e_i}(x) = n\rho_i\bar{\mu}_i = n\bar{\lambda}_i$ . It is evident that the functions $\psi^n_{e_i}$ and $\psi^n_{-e_i}$ satisfy (2.11) and (2.12). Note that $z_i^n(x) \leq x_i$ , and thus Hypothesis 2.1(d) is satisfied. Let ${\mathcal{V}}_{\zeta,m}(x) := \sum_{i\in{\mathscr{I}}}\zeta_i\vert{x_i}\vert^m$ for $x\in\mathbb{R}^d$ , an even integer $m\geq 2$ , and a positive vector $\zeta\in\mathbb{R}^d$ to be chosen later. Recall ${\mathcal{G}}^n_k$ in (2.13). It is straightforward to verify that

\begin{equation*}\begin{aligned}{\mathcal{G}}^n_k{\mathcal{V}}_{\zeta,m}(\hat{x})& = n^{-\beta}\sum_{i\in{\mathscr{I}}} - \phi^n_{-e_i}(n^{\beta}\hat{x}+ n\rho,k)\lambda_i\vert{\hat{x}_i}\vert^{m-1} \\[5pt] &\qquad + n^{-2\beta}\sum_{i\in{\mathscr{I}}}\bigl(2n\bar{\lambda}_i + \phi^n_{-e_i}(n^{\beta}\hat{x}+ n\rho,k)\bigr){\mathscr{O}}(\vert{\hat{x}_i}\vert^{m-2}).\end{aligned}\end{equation*}

Since $\inf_{i,k}\{\mu_i(k),\gamma_i(k)\} > 0$ , it follows by [Reference Arapostathis, Biswas and Pang2, Lemma 5.1] that there exist some positive vector $\lambda$ , $n_0\in\mathbb{N}$ , and positive constants $C_1$ and $C_2$ such that

(3.6) \begin{equation}{\mathcal{G}}^n_k{\mathcal{V}}_{\zeta,m}(\hat{x}) \,\leq\, C_1- C_2{\mathcal{V}}_{\zeta,m}(\hat{x}),\qquad (x,k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\quad n\geq n_0.\end{equation}

Therefore, the result in Corollary 2.1 follows. We remark that the claim in Corollary 2.1 holds for any work-conserving scheduling policy satisfying (3.6), since there is no continuity assumption on $\phi^n_{-e_i}$ . This extends the results of [Reference Arapostathis, Das, Pang and Zheng3, Theorem 4 and Lemma 3]. Indeed the proofs of these results can be greatly simplified by following the approach above, since we only need to consider the constant functions $\psi^n_{e_i}$ and $\psi^n_{-e_i}$ in x.

Next we focus on steady-state approximations for this example. It is straightforward to verify that the coefficients in (2.27) take the form

(3.7) \begin{equation}\begin{aligned}\bar{b}^n_i(x) & = -\frac{\bar{\mu}_i}{n^{\beta}}\bigl(z^n_i(n^{\beta}x + x^n_{*}) - z^n_i(x^n_*)\bigr) \\&\mspace{120mu}- \frac{\bar\gamma_i}{n^{\beta}}\bigl(n^{\beta}x_i- (z^n_i(n^{\beta}x + x^n_{*}) - z^n_i(x^n_*)) \bigr),\qquad i\in{\mathscr{I}},\end{aligned}\end{equation}

and

\begin{equation*}\bar{a}^n_{ii}(0) = \frac{1}{n^{2\beta}}\bigl(n\bar{\lambda}_i + \bar{\mu}_i z^n_i({x}^n_*)+ \bar{\gamma}_i\bigl(x^n_{*,i} - z^n_i(x^n_*)\bigr) = n^{1-2\beta}2\bar{\lambda}_i,\qquad \forall\,i\in{\mathscr{I}},\end{equation*}

and that $\bar{a}^n_{ij}(0) = 0$ for $i\neq j$ . We let ${\mathscr{V}}_{\zeta,m}(x) = \kappa + \sum_{i\in{\mathscr{I}}}\zeta_i\vert{x_i}\vert^m$ for some positive vector $\zeta\in\mathbb{R}^d$ , an even integer $m\geq 2$ , and $\kappa \geq 1$ . We choose $\tilde{\kappa}\geq1$ such that

\begin{equation*}\widetilde{{\mathscr{V}}}_{\zeta,m}(x) \,:=\,\tilde{\kappa}\bigg(1 + \sum_{i\in{\mathscr{I}}}\zeta_i\vert{x}\vert^{6+m}\bigg)\,\geq\, {\mathscr{V}}_{\zeta,m}(x)(1 + \vert{x}\vert^5) \quad\forall\,x\in\mathbb{R}^d.\end{equation*}

Repeating the calculation in [Reference Arapostathis, Biswas and Pang2, Lemma 5.1], we find that there exist some positive vector $\zeta\in\mathbb{R}^d$ and some positive constants $c_1$ and $c_2$ such that

\begin{equation*}\langle \bar{b}^n(x), \nabla{\mathscr{V}}_{\zeta,m}(x)\rangle\,\leq\, c_1 - c_2{\mathscr{V}}_{\zeta,m}(x)\qquad \forall\,x\in\mathbb{R}^d.\end{equation*}

It follows directly by Young’s inequality that there exists some positive constant $c_3$ such that

\begin{equation*}\bigl\vert{\nabla^2{\mathscr{V}}_{\zeta,m}(x)}\bigr\vert \,\leq\, c_3 - \frac{c_2}{2}{\mathscr{V}}_{\zeta,m}(x)\qquad \forall\,x\in\mathbb{R}^d.\end{equation*}

The same holds for $\widetilde{{\mathscr{V}}}_{\zeta,m}$ . Thus, we have verified Assumption 2.3. Since $z^n_i$ is Lipschitz continuous, it is evident that Assumption 2.2 holds. An easy calculation shows that (3.5) holds. As a result, Corollary 2.3 follows.

When $d=1$ , (2.20) becomes

\begin{equation*}\sum_{k\in{\mathscr{K}}}\pi_k \Xi^n({x}^n_*,k) = n\bar{\lambda} - \bar{\mu}\bigl({x}^n_* \wedge n\bigr)- \bar{\gamma}\bigl({x}^n_* - n)^+ = 0,\end{equation*}

which can be solved directly without the critically-loaded assumption. It is straightforward to verify that (3.7) becomes

\begin{equation*}\begin{aligned}\bar{b}^n(x) & = -\bar{\mu}\bigl((x+ n^{-\beta}x^n_*)\wedge n^{1-\beta}- n^{-\beta}x^n_*\wedge n^{1-\beta}\bigr) \\&\mspace{120mu} - \bar{\gamma}\bigl( (x + n^{-\beta}x^n_* - n^{1-\beta})^+-n^{-\beta}(x^n_*-n)^+\bigr).\end{aligned}\end{equation*}

Repeating the procedure as above, we establish Corollary 2.3.

Example 3.3. (Markov-modulated $M/PH/n + M$ queues.) We assume that all customers start service in phase 1, and there are d phases. Given $J^n = k$ , the probability of getting phase j after finishing service in phase i is denoted by $p_{ij}(k)$ . Let $X_1^n$ denote the total number of customers, both in service and queued, in phase 1, and let $X_i^n$ , for $i\neq 1$ , denote the number of customers in service in phase i. (We refer the reader to [Reference Dai, He and Tezcan8] for a detailed description of the model without Markov modulation, and to [Reference Zhu and Prabhu26] for an application of Markov-modulated phase-type distributions in queueing.) Then (2.19) becomes

\begin{equation*}\begin{cases}\Xi^n_1(x,k) = n\lambda(k)- \mu_1(k)\bigl(x_1 - (\langle e, x \rangle - n)^+\bigr)- \gamma(k)\bigl(\langle e,x \rangle - n\bigr)^+, \\[3pt] \begin{aligned}\Xi^n_i(x,k) & = -\mu_i(k)x_i+ \sum_{j\neq i,j\neq 1}p_{ji}(k)\mu_j(k)x_j \\[3pt] &\mspace{200mu} + p_{1i}(k)\mu_1(k)\bigl(x_1 - (\langle e, x \rangle - n)^+\bigr)\quad \text{for}\ i\neq 1,\end{aligned}\end{cases}\end{equation*}

and (2.20) becomes

\begin{align*}\begin{cases}n \bar\lambda- \bar{\mu}_1\bigl(x^n_{*,1} - (\langle e, x^n_* \rangle - n)^+\bigr)- \bar{\gamma}\bigl(\langle e,x^n_* \rangle - n\bigr)^+ = 0,\\[5pt] - \bar{\mu}_ix^n_{*,i}+ \sum_{j\neq i,j\neq 1}\bar{p}_{ji}\bar{\mu}_jx^n_{*,j}+ \bar{p}_{1i}\bar{\mu}_1\bigl(x^n_{*,1} - (\langle e, x^n_* \rangle - n)^+\bigr) = 0\quad \text{for}\ i\neq 1,\end{cases}\end{align*}

where $\bar{\gamma} = \sum_{k\in{\mathscr{K}}}\pi_k\gamma(k)$ , and $\bar{p}_{ij} = \sum_{k\in{\mathscr{K}}}\pi_k p_{ij}(k)$ . Here, $e^{\mathsf{T}}=(1,\dotsc,1)$ as defined in Section 1.2. Assume that $\bar{\lambda} = 1$ . Note that $\{\Xi^n_i: i\in{\mathscr{I}}\}$ are piecewise linear functions in their first argument. It is straightforward to verify that Hypothesis 2.1 and Assumption 2.2 are satisfied. We get $x^n_*= n\rho$ , where

\begin{equation*}\rho \,:=\, \frac{\bar{M}^{-1}e_1}{e^{\mathsf{T}} \bar{M}^{-1}e_1}, \quad \text{and} \quad\bar{M} \,:=\, (I - \bar{P}^{\mathsf{T}})\text{diag}(\bar{\mu}),\end{equation*}

with I the identity matrix and $\bar{P}:= [\bar{p}_{ij}]$ . The coefficients in (2.27) satisfy

\begin{align*}\bar{b}^n(x) & = - \bar{M}x+ \bigl(\bar{M} - \bar{\gamma} I\bigr)e_1\langle e,x \rangle^+,\\[5pt] \bar{a}^n_{ii}(0) & = \begin{cases}n^{1-2\beta}\bigl(1 + \bar{\mu}_1\rho_1\bigr) \quad & \text{if}\ i=1, \\[5pt] n^{1-2\beta}\Bigl(\sum_{j\neq i,j\neq 1}\bar{p}_{ji}\bar{\mu}_j\rho_j + \bar{\mu}_i\rho_i+ \bar{\mu}_1\rho_1\bar{p}_{1i}\Bigr) \quad & \text{if}\ i\neq1,\end{cases}\end{align*}

and

\begin{equation*}\bar{a}^n_{ij}(0) = n^{1-2\beta}\bigl(\bar{p}_{ij}\bar{\mu}_i\rho_i + \bar{p}_{ji}\bar{\mu}_j\rho_j\bigr), \qquad i\neq j.\end{equation*}

By [Reference Arapostathis, Pang and Sandrić5, Theorem 3.5] (see also [Reference Dieker and Gao9, Theorem 3]), there exists a function $\widetilde{{\mathscr{V}}}$ satisfying the assumption in Corollary 2.3. In analogy to [Reference Arapostathis, Pang and Sandrić5, Theorem 3.5], we can show that there exists a function ${\mathcal{V}}(x) = \langle x, Rx \rangle^{m/2}$ , for $m\geq 2$ and some positive definite matrix R, satisfying the conditions in Theorem 2.1.

4. Proofs of Theorem 2.1 and Corollaries 2.1 and 2.2

The range of the transition matrix Q is the subspace $\Delta := \{y\in\mathbb{R}^{k_\circ}\,: \sum_{k\in{\mathscr{K}}} \pi_k y_k=0\}$ . As shown in [Reference Hunter13, Theorem 3.5], if v and u are any vectors in $\mathbb{R}^{k_\circ}$ satisfying $\pi^{\mathsf{T}} v\ne0$ and $u^{\mathsf{T}} e\ne0$ , then the matrix $Q+v u^{\mathsf{T}}$ is nonsingular, and

(4.1) \begin{equation}{\mathcal{T}}:= \bigl(Q+v u^{\mathsf{T}}\bigr)^{-1}\end{equation}

is a generalized inverse of Q; that is, it satisfies $Q{\mathcal{T}} Q=Q$ . This of course means that

(4.2) \begin{equation}Q{\mathcal{T}} y = y\quad\text{for}\ \text{all}\ y\in\Delta.\end{equation}

We also need the following definition.

Definition 4.1. Recall (2.3) and Definition 2.2. Let $\breve{\mathscr{L}}_k^{\kern1.6pt n}:=\overline{\mathscr{L}}^{\,n}-{\mathscr{L}}_k^{\kern1.6pt n}$ . This operator takes the form

\begin{equation*}\breve{\mathscr{L}}_k^{\kern1.6pt n}f(\hat{x},k) \,:=\, \sum_{z\in{\mathscr{Z}}^{\,\,n}}\breve{r}^n_k(n^{\beta}\hat{x} + x^n_*,z)\bigl(f(\hat{x}+n^{-\beta}z,k)-f(\hat{x},k)\bigr),\qquad (\hat{x},k) \in \widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation*}

for $f\in C_b(\mathbb{R}^d\times{\mathscr{K}}\,)$ , where

\begin{equation*}\breve{r}^n_k(x,z) \,:=\,\bar{r}^n(x,z) - \tilde{r}^n_k(x,z),\qquad (x,k)\in {{\mathfrak{X}}}^n\times{\mathscr{K}}.\end{equation*}

Proof of Theroem 2.1. Let ${\mathcal{T}}=[{\mathcal{T}}_{k\ell}]_{k,\ell\in{\mathscr{K}}}$ be as defined in (4.1).

(4.3) \begin{equation}\widetilde{{\mathcal{V}}}^n(\hat{x},k) \,:=\,\frac{1}{n^{\alpha}}\sum_{\ell\in{\mathscr{K}}}{\mathcal{T}}_{k\ell}\,\breve{\mathscr{L}}^{\,n}_{\ell}{\mathcal{V}}^n(\hat{x}),\qquad (\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}.\end{equation}

Then

(4.4) \begin{equation}{\mathcal{Q}}^n\widetilde{{\mathcal{V}}}^n(\hat{x},k) = \breve{\mathscr{L}}_k^{\kern1.6pt n}{\mathcal{V}}^n(\hat{x}) \qquad\forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation}

by (4.2).

We define

(4.5) \begin{equation}\widehat{{\mathcal{V}}}^n(\hat{x},k)\,:=\, {\mathcal{V}}^n(\hat{x}) + \widetilde{{\mathcal{V}}}^n(\hat{x},k),\qquad(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}.\end{equation}

By Hypothesis 2.1(c)–(d), we have

(4.6) \begin{equation}\tilde{r}^n_k(n^{\beta}\hat{x} + x^n_*,z)\,\leq\, C_0 (n^{1\vee\alpha/2}+n^{\beta}\vert{\hat{x}}\vert)\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\ \forall\,z\in{\mathscr{Z}}^{\,\,n}, \ \forall\,n\in\mathbb{N}.\end{equation}

We choose $N_1$ large enough so that $m_0 \leq \varepsilon_0 N_1^\beta$ , with $m_0$ as defined in Hypothesis 2.1(a). By Hypothesis 2.1(a)–(b), (2.6), and (4.6), we have

(4.7) \begin{equation}\bigl\vert{\breve{\mathscr{L}}^{\,n}_{k} {\mathcal{V}}^n(\hat{x})}\bigr\vert\,\leq\, N_0 C_0 (n^{1\vee\alpha/2} +n^{\beta}\vert{\hat{x}}\vert)\, C m_0\frac{1+{\mathcal{V}}^n(\hat{x})}{n^\beta (1+\vert{\hat{x}}\vert)}\end{equation}

for all $n\geq N_1$ . Therefore, since $\alpha + \beta - 1 \geq \alpha/2$ for $\alpha > 0$ , it follows by (4.5)–(4.7) that there exists $n_1\in\mathbb{N}$ , $n_1\geq N_1$ , such that (2.8) holds.

Recall the definitions in (2.2), (2.3), and (2.5). We have

\begin{equation*}\overline{{\mathscr{L}}}^n{\mathcal{V}}^n(\hat{x}) = {\mathscr{L}}_k^n{\mathcal{V}}^n(\hat{x}) + \breve{\mathscr{L}}_k^{\kern1.6pt n}{\mathcal{V}}^n(\hat{x}) = {\mathscr{L}}_k^n{\mathcal{V}}^n(\hat{x}) + {\mathcal{Q}}^n \widetilde{{\mathcal{V}}}^n(\hat{x},k)\end{equation*}

by (4.4). Therefore, since ${\mathcal{Q}}^n {\mathcal{V}}^n(\hat{x}) = 0$ , we obtain

(4.8) \begin{equation}\begin{aligned}\widehat{{\mathscr{\!\!L}}}^{\,\,n} \widehat{{\mathcal{V}}}^n(\hat{x},k) & = {\mathscr{L}}_k^{\kern1.6pt n}{\mathcal{V}}^n(\hat{x}) + {\mathscr{L}}_k^{\kern1.6pt n} \widetilde{{\mathcal{V}}}^n(\hat{x},k)+ {\mathcal{Q}}^n \widetilde{{\mathcal{V}}}^n(\hat{x},k) \\[4pt] & = \,\,\overline{{\mathscr{\!\!L}}}^{\,\,n}{\mathcal{V}}^n(\hat{x})+ {\mathscr{L}}_k^{\kern1.6pt n} \widetilde{{\mathcal{V}}}^n(\hat{x},k)\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^{n}\times{\mathscr{K}}.\end{aligned}\end{equation}

We define the function

\begin{equation*}G^n_k(\hat{x},z) \,:=\,\breve{r}^n_k\bigl(n^{\beta}\hat{x} + x^n_{*},z\bigr)\bigl({\mathcal{V}}^n(\hat{x}+n^{-\beta}z) - {\mathcal{V}}^n(\hat{x})\bigr).\end{equation*}

It is straightforward to verify, using (4.3), that

(4.9) \begin{equation}\begin{aligned}{\mathscr{L}}_k^{\kern1.6pt n} \widetilde{{\mathcal{V}}}^n(\hat{x},k)\,=&\, \sum_{h\in{\mathscr{Z}}^{\,\,n}}\tilde{r}^n_k(n^{\beta}\hat{x} + {x}^n_*,h)\bigl(\widetilde{{\mathcal{V}}}^n(\hat{x} + n^{-\beta}h,k)- \widetilde{{\mathcal{V}}}^n(\hat{x},k)\bigr)\\[4pt] \,=&\, \frac{1}{n^{\alpha}} \sum_{h,z\in{\mathscr{Z}}^{\,\,n}}\tilde{r}^n_{k}(n^{\beta}\hat{x} + {x}^n_*,h) \sum_{\ell\in{\mathscr{K}}}{\mathcal{T}}_{k\ell}\bigl(G^n_{\ell}(\hat{x} + n^{-\beta}h,z) - G^n_{\ell}(\hat{x},z)\bigr).\end{aligned}\end{equation}

On the other hand, it follows by Hypothesis 2.1(c) and a triangle inequality that

(4.10) \begin{equation}\begin{aligned}&\vert{G^n_k(\hat{x} + n^{-\beta}h,z) - G^n_k(\hat{x},z)}\vert \\ &\mspace{100mu}\,\leq\,2 C_0(n^{\alpha/2}+\vert{h}\vert)\,\bigl\vert{{\mathcal{V}}^n(\hat{x}+n^{-\beta}z) - {\mathcal{V}}^n(\hat{x})}\bigr\vert \\ &\mspace{150mu}+\bigl\vert{\breve{r}^n_k(n^{\beta}\hat{x} + x^n_{*} + h,z)}\bigr\vert\bigl\vert{{\mathcal{V}}^n(\hat{x} + n^{-\beta}z + n^{-\beta}h)} \\ &\mspace{200mu}- {\mathcal{V}}^n(\hat{x} + n^{-\beta}h) - {\mathcal{V}}^n(\hat{x}+ n^{-\beta}z)+ {\mathcal{V}}^n(\hat{x})\bigr\vert\end{aligned}\end{equation}

for all $h,z\in{\mathscr{Z}}^{\,\,n}$ . As in (4.6), we have

(4.11) \begin{equation}\vert{\breve{r}^n_k(n^{\beta}\hat{x} + x^n_*+h,z)}\vert\,\leq\, C_0 (n^{1\vee\alpha/2}+n^{\beta}\vert{\hat{x}}\vert + \vert{h}\vert)\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\ \forall\,h,z\in{\mathscr{Z}}^{\,\,n},\end{equation}

for all $n\in\mathbb{N}$ . By (2.6) and Hypothesis 2.1(a), we have

(4.12) \begin{equation}\begin{aligned}\bigl\vert{{\mathcal{V}}^n(\hat{x}+n^{-\beta}z) - {\mathcal{V}}^n(\hat{x})}\bigr\vert&\,\leq\, C m_0\frac{1+{\mathcal{V}}^n(\hat{x})}{n^\beta (1+\vert{\hat{x}}\vert)},\\[3pt]\bigl\vert{{\mathcal{V}}^n(\hat{x} + n^{-\beta}z + n^{-\beta}h)- {\mathcal{V}}^n(\hat{x} + n^{-\beta}h)\mspace{50mu}}&\\- {\mathcal{V}}^n(\hat{x}+ n^{-\beta}z) + {\mathcal{V}}^n(\hat{x})\bigr\vert&\,\leq\, C m_0^2\frac{1+{\mathcal{V}}^n(\hat{x})}{n^{2\beta} (1+\vert{\hat{x}}\vert^2)}\end{aligned}\end{equation}

for all $h,z\in B_{m_0}$ , $\hat{x}\in\widehat{\mathfrak{X}}^n$ , and $n\in\mathbb{N}$ . Hence, using (4.9) together with the estimates in (4.6) and (4.10)–(4.12) and Hypothesis 2.1(a)–(b), we obtain

(4.13) \begin{equation}\begin{aligned}{\mathscr{L}}_k^{\kern1.6pt n} & \widetilde{{\mathcal{V}}}^n(\hat{x},k)\\ &\,\leq\,N_0C_0C m_0\sum_{k,k^{\prime}\in{\mathscr{K}}}\vert{{\mathcal{T}}_{k\ell}}\vert\biggl(2(n^{\alpha/2} + m_0)(n^{1\vee\alpha/2} +n^{\beta}\vert{\hat{x}}\vert)\frac{1+{\mathcal{V}}^n(\hat{x})}{n^{\alpha+\beta} (1+\vert{\hat{x}}\vert)}\\&\mspace{40mu}+N_0C_0m_0 (n^{1\vee\alpha/2}+n^{\beta}\vert{\hat{x}}\vert)(n^{1\vee\alpha/2} +n^{\beta}\vert{\hat{x}}\vert + m_0)\frac{1+{\mathcal{V}}^n(\hat{x})}{n^{\alpha+2\beta} (1+\vert{\hat{x}}\vert^2)}\biggr).\end{aligned}\end{equation}

Using the property $\beta=\max\{1/2,1 - \alpha/2\}$ , we deduce from (4.13) that for any $\epsilon>0$ there exists some constant $C_\circ(\epsilon)$ such that

(4.14) \begin{equation}{\mathscr{L}}_k^{\kern1.6pt n} \widetilde{{\mathcal{V}}}^n(\hat{x},k)\,\leq\, C_\circ(\epsilon) + \epsilon {\mathcal{V}}^n(\hat{x})\qquad\forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\ \forall n\in\mathbb{N}.\end{equation}

Therefore, choosing $\epsilon=\frac{1}{2}\overline{C}_2$ , and using (2.7), (2.8), (4.8), and (4.14), we obtain

\begin{equation*}\widehat{{\mathscr{\!\!L}}}^{\,\,n} \widehat{{\mathcal{V}}}^n(\hat{x},k) \,\leq\, \overline{C}_1+ C_\circ\bigl(\overline{C}_2/2\bigr)+\frac{1}{6}\overline{C}_2 - \frac{1}{3}\overline{C}_2\widehat{{\mathcal{V}}}^n(\hat{x},k) \qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\ \forall\, n>n_1.\end{equation*}

This completes the proof.

Proof of Corollary 2.1. Recall ${\mathcal{G}}^n_k$ in (2.13), and let $\breve{\mathcal{G}}^n_k := {\mathcal{G}}^n_k - {\mathscr{L}}_k^{\kern1.6pt n}$ . Then $\breve{\mathcal{G}}^n_k$ takes the form

\begin{equation*}\breve{\mathcal{G}}^n_k f(\hat{x},k) = \sum_{z\in{\mathscr{Z}}^{\,\,n}} \breve\psi^n_k(n^\beta\hat{x} + x^n_*,z)\bigl(f(\hat{x}+n^{-\beta}z,k)-f(\hat{x},k)\bigr),\qquad (\hat{x},k) \in \widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation*}

for $f\in C_b(\mathbb{R}^d\times{\mathscr{K}}\,)$ , where

\begin{equation*}\breve{\psi}^n_k(x,z)\,:=\,\bar{\psi}^n(x,z) - \psi^n_k(x,z), \qquad k\in{\mathscr{K}}, \quad (x,z)\in{\mathfrak{X}}^n\times{\mathscr{Z}}^{\,\,n}.\end{equation*}

Compare this to Definition 4.1. We let

\begin{equation*}\widetilde{{\mathcal{V}}}^n(\hat{x},k) \,:=\,\frac{1}{n^{\alpha}}\sum_{\ell\in{\mathscr{K}}}{\mathcal{T}}_{k\ell}\,\breve{\mathcal{G}}^n_{\ell}{\mathcal{V}}^n(\hat{x}),\qquad (\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}.\end{equation*}

As in (4.4), we have

\begin{equation*}{\mathcal{Q}}^n\widetilde{{\mathcal{V}}}^n(\hat{x},k) = \breve{\mathcal{G}}^n_k{\mathcal{V}}^n(\hat{x}), \qquad(\hat{x},k)\in\widehat{{\mathfrak{X}}}^{n}\times{\mathscr{K}}.\end{equation*}

In analogy to (4.8), we get

\begin{equation*}\widehat{\mathscr{\!\!L}}^{\,\,n} \widehat{{\mathcal{V}}}^n(\hat{x},k) = {\mathcal{G}}^n_k{\mathcal{V}}^n(\hat{x})+ {\mathscr{L}}_k^{\kern1.6pt n} \widetilde{{\mathcal{V}}}^n(\hat{x},k),\qquad (\hat{x},k)\in\widehat{{\mathfrak{X}}}^{n}\times{\mathscr{K}}.\end{equation*}

In obtaining an estimate for ${\mathscr{L}}_k^{\kern1.6pt n} \widetilde{{\mathcal{V}}}^n(\hat{x},k)$ , the proof is the same as that of Theorem 2.1, except that we replace $\breve r^n$ with $\breve\psi^n$ , and use (2.11) and (2.12). Applying (2.11) and (2.12) again, we may show (2.8). Then the claim in (2.9) follows by (2.14).

Proof of Corollary 2.2. We present only some crucial estimates that are different from those in the proof of Theorem 2.1. Indeed, it follows by (2.15) and (2.16) that

(4.15) \begin{equation}\tilde{r}^n_k(n^{\beta}\hat{x} + x^n_*,z) \,\leq\, C_0(1 + n),\qquad \breve{r}^n_k(n^{\beta}\hat{x} + x^n_*,z) \,\leq\, C_0(1 + n),\end{equation}

and

(4.16) \begin{align}\bigl\vert{\breve{r}^n_k(n^{\beta}(\hat{x} + n^{-\beta}h) + x^n_*,z) - \breve{r}^n_k(n^{\beta}\hat{x} + x^n_*,z)} \bigr\vert\,\leq\, C_0(1 + \vert{h}\vert\wedge n)\end{align}

for some positive constant $C_0$ . By Hypothesis 2.1(a)–(b), (4.15), and (2.17), (4.7) becomes

(4.17) \begin{equation}\bigl\vert{\breve{\mathscr{L}}^{\,n}_{k} {\mathcal{V}}^n(\hat{x})}\bigr\vert\,\leq\, N_0 C_0 (1 + n)\, C m_0\frac{1+{\mathcal{V}}^n(\hat{x})}{n^\beta}\end{equation}

for all large n. Using (2.17) and (4.15)–(4.17), together with Hypothesis 2.1(a)–(b), (4.13) becomes

(4.18) \begin{equation}\begin{aligned}{\mathscr{L}}_k^{\kern1.6pt n} \widetilde{{\mathcal{V}}}^n(\hat{x},k) &\,\leq\,N_0C_0C m_0\sum_{k,k^{\prime}\in{\mathscr{K}}}\vert{{\mathcal{T}}_{k\ell}}\vert\biggl(2(1 + m_0)(1+n)\frac{1+{\mathcal{V}}^n(\hat{x})}{n^{\alpha+\beta}}\\&\mspace{175mu}+N_0C_0m_0 (1+n)(1+n)\frac{1+{\mathcal{V}}^n(\hat{x})}{n^{\alpha+2\beta}}\biggr).\end{aligned}\end{equation}

Since $\alpha + 2\beta > 2$ implies $\alpha + \beta >1$ , it follows by (4.18) that (4.14) holds for all large n. The rest of the proof is the same as that of Theorem 2.1.

5. Proofs of Theorem 2.2 and Corollary 2.3

We need to introduce some additional notation to facilitate the proofs. Recall the definitions of $\widehat{\Xi}^n$ , $\bar{\Gamma}^n$ , $\bar{b}^n$ , and $\bar{a}^n$ in (2.25)–(2.27) and (2.30), respectively. For $f\in C^2(\mathbb{R}^{d})$ and $n\in\mathbb{N}$ , let

(5.1) \begin{equation}\begin{aligned}\breve{g}^n_1[f](x,k)&\,:=\, \sum_{i\in{\mathscr{I}}}\bigl(\bar{b}^n_i(x) -\bigl(\widehat{\Xi}^n_i(x,k) - \widehat{\Xi}^n_i(0,k) \bigr)\bigr)\partial_i f(x) \\&\mspace{180mu}+ \frac{1}{2}\sum_{i,j\in{\mathscr{I}}}\bigl(\bar{a}^n_{ij}(x)- \bar{\Gamma}^n_{ij}(x,k)\bigr)\partial_{ij}f(x)\end{aligned}\end{equation}

and

(5.2) \begin{equation}\begin{aligned}\breve{g}^n_2[f](x,k)&\,:=\, \frac{1}{n^{\alpha + 2\beta}}\sum_{z}\sum_{h\in{\mathscr{K}}}\biggl(\sum_{l\in{\mathscr{K}}}\pi_l\xi^n_z(n^{\beta}x + {x}^n_{*},l)\Upsilon_{lh} \\&\mspace{100mu}- \xi^n_z(n^{\beta}x + {x}^n_{*},k)\Upsilon_{kh}\biggr)\sum_{j\in{\mathscr{I}}}\Xi^n_j({x}^n_{*},h)\sum_{i\in{\mathscr{I}}}z_i\partial_{ij} f(x),\end{aligned}\end{equation}

with $\Upsilon$ as defined in (2.28). It follows by the identity

\begin{equation*}\sum_{k\in{\mathscr{K}}}\Bigl(\sum_{l\in{\mathscr{K}}}\pi_l\xi^n_z(n^{\beta}x + {x}^n_{*},l)\Upsilon_{lh}- \xi^n_z(n^{\beta}x + {x}^n_{*},k)\Upsilon_{kh}\Bigr)\,\equiv\,0\end{equation*}

that $\sum_{k\in{\mathscr{K}}}\pi_k \breve{g}^n_2[f](x,k) = 0$ . It is clear that $\sum_{k\in{\mathscr{K}}}\pi_k \breve{g}^n_1[f](x,k) = 0$ . Recall the matrix ${\mathcal{T}}$ in (4.1) and (4.2). We define

(5.3) \begin{equation}g^n_{i}[f](x,k) \,:=\, \frac{1}{n^{\alpha}}\sum_{\ell\in{\mathscr{K}}}{\mathcal{T}}_{k\ell}\, \breve{g}^n_i[f](x,\ell),\qquad i=1,2,\end{equation}

and thus

(5.4) \begin{equation}{\mathcal{Q}}^ng^n_i[f](x,k) = \breve{g}^n_i[f](x,k),\qquad i=1,2.\end{equation}

For $f\in C^2(\mathbb{R}^{d})$ and $n\in\mathbb{N}$ , let

(5.5) \begin{equation}g^n_{3}[f](x,k)\,:=\, \frac{1}{n^{\alpha+\beta}}\sum_{h\in{\mathscr{K}}}\sum_{j\in{\mathscr{I}}}\Xi^n_j({x}^n_{*},h)\Upsilon_{kh}\,\partial_jf(x).\end{equation}

Note that the function $g^n_3[f]$ corresponds to the covariance of the background Markov process $J^n$ . We let $g^n[f]$ denote the sum of the above functions, that is,

(5.6) \begin{equation}g^n[f](x,k) \,:=\, g^n_{1}[f](x,k)+ g^n_{2}[f](x,k) + g^n_{3}[f](x,k),\qquad (x,k)\in\mathbb{R}^d\times{\mathscr{K}}.\end{equation}

To keep the algebraic expressions in the proofs manageable, we adopt the notation introduced in the following definition.

Definition 5.1. We define the operators $[{\mathcal{D}}^n_z]^{0}$ and $[{\mathcal{D}}^n_z]^{1}_j$ , $j\in{\mathscr{I}}$ , by

\begin{align*}[{\mathcal{D}}^n_z]^{0}f(x) & := f(x+n^{-\beta}z) - f(x)- n^{-\beta}\sum_{i\in{\mathscr{I}}}z_i\partial_if(x)- n^{-2\beta}\sum_{i,j\in{\mathscr{I}}}z_{i}z_{j}{\partial_{ij}}\,f(x),\\ \,[\,{\mathcal{D}}^{n}_{z}\,]^{1}_{j}f(x) & := \partial_jf(x+n^{-\beta}z) - \partial_jf(x)- n^{-\beta}\sum_{i\in{\mathscr{I}}}z_{i}\partial_{ij}\, f(x),\end{align*}

for $f\in C^2(\mathbb{R}^d)$ and $z\in{\mathscr{Z}}^{\,\,n}$ . In addition, we define

\begin{equation*}\begin{aligned}{\mathcal{R}}^n_1[f](\hat{x},k) &\,:=\, \sum_{z}^{}\xi^n_{z}(n^{\beta}\hat{x}+ {x}^n_{*},k)\,[{\mathcal{D}}^n_z]^{0}f(\hat{x}),\\{\mathcal{R}}^n_2[f](\hat{x}) &\,:=\, \frac{1}{2}\sum_{i,j\in{\mathscr{I}}}\sum_{k\in{\mathscr{K}}}\pi_k\bigl(\bar{\Gamma}^n_{ij}(\hat{x},k) - \bar{\Gamma}^n_{ij}(0,k)\bigr)\partial_{ij}f(\hat{x}),\\{\mathcal{R}}^n_3[f](\hat{x},k) &\,:=\, \frac{1}{n^{\alpha+2\beta}}\sum_{i,j\in{\mathscr{I}}}\sum_{h,l\in{\mathscr{K}}}\bigl(\Xi^n_i({x}^n_* + n^{\beta}\hat{x},l) - \Xi^n_i({x}^n_*,l)\bigr)\Xi^n_j({x}^n_{*},h)\pi_l\Upsilon_{lh}\partial_{ij}f(\hat{x}),\\{\mathcal{R}}^n_4[f](\hat{x},k) &\,:=\, \frac{1}{n^{\alpha+\beta}}\sum_{z}\sum_{h\in{\mathscr{K}}}\xi^n_z(n^{\beta}\hat{x} + {x}^n_{*},k)\Upsilon_{kh}\sum_{j\in{\mathscr{I}}}\Xi^n_j({x}^n_{*},h)[{\mathcal{D}}^n_z]^{1}_jf(\hat{x}),\\{\mathcal{R}}^n_5[f](\hat{x},k) &\,:=\, {\mathscr{L}}_k^{\kern1.6pt n}\,g^n_{1}[f](\hat{x},k),\\[3pt]{\mathcal{R}}^n_6[f](\hat{x},k) &\,:=\, {\mathscr{L}}^{\,n}_{k}\,g^n_{2}[f](\hat{x},k).\end{aligned}\end{equation*}

The following lemma establishes a useful identity involving the generator of $(\widehat{X}^n,J^n)$ in (2.2) and that of $\widehat{Y}^n$ in (2.31) and the operators ${\mathcal{R}}^n_i$ in Definition 5.1.

Lemma 5.1. Under Assumption 2.2(ii), for $f\in C^2(\mathbb{R}^d)$ , we have

(5.7) \begin{equation}\widehat{\mathscr{\!\!L}}^{\,\,n} f(\hat{x}) + \widehat{\mathscr{\!\!L}}^{\,\,n} g^n[f](\hat{x},k) = {\mathscr{A}}^n f(\hat{x}) + \sum_{i=1}^6 {\mathcal{R}}^n_i[f](\hat{x},k),\qquad (\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}.\end{equation}

Proof. By (2.2) we have

(5.8) \begin{equation}\widehat{\mathscr{\!\!L}}^{\,\,n} g^n[f](\hat{x},k) = \sum_{i=1}^3\Bigl({\mathscr{L}}_k^{\kern1.6pt n}g^n_{i}[f](\hat{x},k)+ {\mathcal{Q}}^ng^n_{i}[f](\hat{x},k)\Bigr),\end{equation}

and $\,\,\widehat{\mathscr{\!\!L}}^{\,\,n}f(\hat{x}) = {\mathscr{L}}_k^{\kern1.6pt n} f(\hat{x})$ for any $f\in C^2(\mathbb{R}^d)$ .

We first show that

(5.9) \begin{equation}\begin{aligned}&{\mathscr{L}}_k^{\kern1.6pt n}f(\hat{x}) + {\mathcal{Q}}^ng^n_{1}[f](\hat{x},k)+ {\mathcal{Q}}^ng^n_{3}[f](\hat{x},k) \\&\mspace{80mu} = \sum_{i\in{\mathscr{I}}}\bar{b}^n_i(\hat{x}) \partial_if(\hat{x})+ \frac{1}{2}\sum_{i,j\in{\mathscr{I}}}\bar{a}^n_{ij}\partial_{ij}f(\hat{x})+ {\mathcal{R}}^n_1[f](\hat{x},k) + {\mathcal{R}}^n_2[f](\hat{x}).\end{aligned}\end{equation}

Using (2.3) and (5.5), we obtain

(5.10) \begin{equation}{\mathcal{Q}}^ng^n_{3}[f](\hat{x},k) = \sum_{h\in{\mathscr{K}}}\sum_{\ell\in{\mathscr{K}}} q_{k\ell}\Upsilon_{\ell h}\sum_{j\in{\mathscr{I}}}\frac{\Xi^n_j({x}^n_*,h)}{n^\beta}\partial_jf(\hat{x}).\end{equation}

Since $Q\Upsilon = \Pi - I$ , where I denotes the identity matrix, it follows by (2.20) that

(5.11) \begin{equation}\sum_{h\in{\mathscr{K}}}\sum_{\ell\in{\mathscr{K}}}q_{k\ell}\Upsilon_{\ell h}\Xi^n_j({x}^n_{*},h) = \sum_{h\in{\mathscr{K}}}\pi_h\Xi^n_j({x}^n_*,h) - \Xi^n_j({x}^n_*,k) = - \Xi^n_j({x}^n_*,k),\end{equation}

where in the second equality we use Assumption 2.2(ii). Thus, by (5.10) and (5.11), we have

(5.12) \begin{equation}{\mathcal{Q}}^ng^n_{3}[f](\hat{x},k) = \sum_{j\in{\mathscr{I}}} -\frac{\Xi^n_j({x}^n_*,k)}{n^{\beta}}\,\partial_jf(\hat{x}) = \sum_{j\in{\mathscr{I}}} -\widehat{\Xi}^n_j(0,k)\,\partial_jf(\hat{x}).\end{equation}

By (2.3) and a standard identity, we obtain

(5.13) \begin{equation}\begin{aligned}{\mathscr{L}}_k^{\kern1.6pt n}f(\hat{x}) & = \sum_{z\in{\mathscr{Z}}^{\,\,n}}\xi^n_z(n^{\beta}\hat{x}+ x^n_*,k)\biggl(\sum_{i\in{\mathscr{I}}}n^{-\beta}z_i\partial_i f(\hat{x}) \\ &\mspace{200mu}+ \sum_{i,j\in{\mathscr{I}}}n^{-2\beta}z_iz_j\partial_{ij}f(\hat{x})+ [{\mathcal{D}}^n_z]^0 f(\hat{x})\biggr)\\[5pt] & = \sum_{i\in{\mathscr{I}}}\widehat{\Xi}^n_i(\hat{x},k)\partial_if(\hat{x}) +\sum_{i,j\in{\mathscr{I}}} \bar{\Gamma}^n_{ij}(\hat{x},k) \partial_{ij}f(\hat{x})+ {\mathcal{R}}^n_1[f](\hat{x},k).\end{aligned}\end{equation}

Thus (5.9) follows from (5.1), (5.4), (5.12), and (5.13).

Next, we show that

(5.14) \begin{equation}{\mathscr{L}}_k^{\kern1.6pt n}\,g^n_{3}[f](\hat{x},k) + {\mathcal{Q}}^ng^n_{2}[f](\hat{x},k) = \frac{1}{2}\sum_{i,j\in{\mathscr{I}}}\theta^n_{ij}\partial_{ij}f(\hat{x})+ {\mathcal{R}}^n_3[f](\hat{x},k) + {\mathcal{R}}^n_4[f](\hat{x},k).\end{equation}

We have

\begin{equation*}\begin{aligned}{\mathscr{L}}_k^{\kern1.6pt n} & \,g^n_3[f](\hat{x},k) \\& = \frac{1}{n^{\alpha + \beta}}\sum_{z}\xi^n_{z}(n^{\beta}\hat{x} + {x}^n_{*},k)\sum_{h\in{\mathscr{K}}}\sum_{j\in{\mathscr{I}}}\Xi^n_j({x}^n_{*},h)\Upsilon_{kh}\,\bigl(\partial_jf(\hat{x} + n^{-\beta}z) - \partial_jf(\hat{x})\bigr)\end{aligned}\end{equation*}

by (2.3). It is clear that

\begin{equation*}\partial_jf(\hat{x}+n^{-\beta}z) - \partial_jf(\hat{x}) = n^{-\beta}\sum_{i\in{\mathscr{I}}}z_i\partial_{ij} f(\hat{x}) +[{\mathcal{D}}^n_z]^{1}_jf(\hat{x})\end{equation*}

and

\begin{equation*}\sum_{z}z_i\xi^n_{z}(n^{\beta}\hat{x} + {x}^n_{*},k) = \Xi^n_i({x}^n_*,k)+ \bigl(\Xi^n_i({x}^n_* + n^{\beta}\hat{x},k) - \Xi^n_i({x}^n_*,k)\bigr).\end{equation*}

Therefore, (5.14) follows from combining these identities with (5.2) and (5.4).

Hence, we obtain (5.7) by adding (5.8), (5.9), and (5.14), and using the definitions of ${\mathcal{R}}^n_i[f]$ for $i=5,6$ . This completes the proof.

The following lemma provides needed estimates for ${\mathcal{R}}^n_5$ and ${\mathcal{R}}^n_6$ .

Lemma 5.2. Under Assumption 2.2 (i)–(iii), there exists some positive constant C such that

(5.15) \begin{equation}\begin{aligned}&\bigl\vert{{\mathcal{R}}^n_5[f](\hat{x},k)}\bigr\vert\\[4pt] &\,\leq\,C \biggl[\biggl(\frac{1}{n^{\alpha}}\vert{\hat{x}}\vert+ \frac{1}{n^{\alpha+\beta -1}} \biggr) \vert{\nabla f(\hat{x})}\vert+ \biggl(\frac{1}{n^{\alpha+\beta}}\vert{\hat{x}}\vert + \frac{1}{n^{\alpha+2\beta-1}} \biggr)\bigl\vert{\nabla^2 f(\hat{x})}\bigr\vert\\[4pt] &\quad + \biggl(\frac{1}{n^{\alpha - \beta}}\vert{\hat{x}}\vert^2+ \frac{1}{n^{\alpha-1}}\vert{\hat{x}}\vert\biggr)\,\max_{z\in{\mathscr{Z}}^{\,\,n}}\vert{\nabla f(\hat{x} + n^{-\beta}z) - \nabla f(\hat{x})}\vert\\[4pt] &\quad + \biggl(\frac{1}{n^{\alpha}}\vert{\hat{x}}\vert^2+ \frac{1}{n^{\alpha+\beta-1}}\vert{\hat{x}}\vert+ \frac{1}{n^{\alpha + 2\beta - 2}}\biggr)\,\max_{z\in{\mathscr{Z}}^{\,\,n}}\bigl\vert{ \nabla^2 f(\hat{x}+n^{-\beta}z) - \nabla^2 f(\hat{x})}\bigr\vert\biggr]\end{aligned}\end{equation}

and

(5.16) \begin{equation}\begin{aligned}\bigl\vert{{\mathcal{R}}^n_6[f](\hat{x},k)}\bigr\vert &\,\leq\, C\biggl[\biggl(\frac{1}{n^{2\alpha+\beta -1}}\vert{\hat{x}}\vert+ \frac{1}{n^{2\alpha+2\beta-2}}\biggr)\,\bigl\vert{\nabla^2 f(\hat{x})}\bigr\vert \\[4pt] &\mspace{40mu} +\biggl(\frac{1}{n^{2\alpha - 1}}\vert{\hat{x}}\vert^2+ \frac{1}{n^{2\alpha + \beta -2}}\vert{\hat{x}}\vert \\[4pt] &\mspace{100mu}+ \frac{1}{n^{2\alpha + 2\beta - 3}}\biggr)\max_{z\in{\mathscr{Z}}^{\,\,n}}\,\bigl\vert{\nabla^2 f(\hat{x}+n^{-\beta}z) - \nabla^2 f(\hat{x})}\bigr\vert\biggr],\end{aligned}\end{equation}

for any $(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}$ .

Proof. Recall the functions $g^n_1[f]$ and $g^n_{2}[f]$ in (5.3). It follows by (2.21) and (2.22) that

(5.17) \begin{align}\vert{\xi^n_z(x, k)}\vert \,\leq\,\widetilde{C}(\vert{x - x^n_*}\vert + n)\end{align}

and

(5.18) \begin{align}\vert{\xi^{n}_z(x+x^{n}_*,k) - \xi^n_z(x^{n}_*,k)}\vert \,\leq\, \widetilde{C}\vert{x}\vert\end{align}

for $(x,k)\in\mathbb{R}^d\times{\mathscr{K}}$ , $z\in{\mathscr{Z}}^{\,\,n}$ , and $n\in\mathbb{N}$ . By Assumption 2.2(i), and applying (2.20) and (5.18) it is straightforward to verify that

(5.19) \begin{equation}\Biggl| \sum_{k\in{\mathscr{K}}} \pi_k \widehat\Xi^n(\hat{x},k) \Biggr|\,\leq\, \widetilde{C}\widetilde{N}_0m_0\vert{\hat{x}}\vert\qquad \forall\,\hat{x}\in\mathbb{R}^{d}.\end{equation}

Thus, by (5.18) and (5.19),we have

(5.20) \begin{equation}\bigl\vert{\bar{b}^n(\hat{x}) - \bigl(\widehat{\Xi}^n(\hat{x},k)- \widehat{\Xi}^n(0,k)\bigr)}\bigr\vert\,\leq\, 2\widetilde{C}\widetilde{N}_0m_0\vert{\hat{x}}\vert\qquad \forall\,(\hat{x},k)\in\mathbb{R}^d\times{\mathscr{K}}.\end{equation}

Applying (5.17), we obtain

(5.21) \begin{equation}\bigl\vert{\bar{a}^n(\hat{x}) - \bar{\Gamma}^n(\hat{x},k)}\bigr\vert\,\leq\, 2\widetilde{C}\widetilde{N}_0m_0^2\bigl(n^{-\beta}\vert{\hat{x}}\vert + n^{1- 2\beta}\bigr)\qquad \forall (\hat{x},k)\in\mathbb{R}^d\times{\mathscr{K}},\end{equation}

and

(5.22) \begin{equation}\Biggl| \sum_{l\in{\mathscr{K}}}\pi_l\xi^n_z(n^{\beta}\hat{x} + {x}^n_{*},l)\Upsilon_{lh}- \xi^n_z(n^{\beta}\hat{x} + {x}^n_{*},k)\Upsilon_{kh} \Biggr|\,\leq\, C_1\bigl(n^{\beta}\vert{\hat{x}}\vert + n\bigr) \qquad\forall\, \hat{x}\in\mathbb{R}^d\end{equation}

and all $k,h\in{\mathscr{K}}$ and $z\in{\mathscr{Z}}^{\,\,n}$ , for some positive constant $C_1$ . We have

(5.23) \begin{equation}\vert{\Xi^n(x^n_*,k)}\vert \,\leq\, \widetilde{C}\widetilde{N}_0m_0n\qquad \forall\,k\in{\mathscr{K}},\ n\in\mathbb{N},\end{equation}

by (2.22), and

(5.24) \begin{equation}\vert{\xi^n_z(n^{\beta}\hat{x} + x^n_*,k)}\vert \,\leq\,\widetilde{C}(n^{\beta}\vert{\hat{x}}\vert +n) \qquad\forall\,(\hat{x},k) \in\mathbb{R}^d\times{\mathscr{K}},\ z\in{\mathscr{Z}}^{\,\,n},\ n\in\mathbb{N},\end{equation}

by (5.17). From (2.21), we obtain

(5.25) \begin{equation}\bigl\vert{\widehat{\Xi}^n(\hat{x} + n^{-\beta}z,k) - \widehat{\Xi}^n(\hat{x},k)}\bigr\vert\,\leq\, n^{-\beta}\widetilde{C}\widetilde{N}_0m_0^2\end{equation}

and

(5.26) \begin{align}\bigl\vert{\bar{\Gamma}^n(\hat{x} + n^{-\beta}z,k) - \bar{\Gamma}^n(\hat{x},k)}\bigr\vert\,\leq\, n^{-2\beta}\widetilde{C}\widetilde{N}_0m_0^3\end{align}

for $(\hat{x},k)\in\mathbb{R}^d\times{\mathscr{K}}$ , $z\in{\mathscr{Z}}^{\,\,n}$ , and $n\in\mathbb{N}$ . Repeating similar calculations as in (4.10) and (4.13), and applying (5.20), (5.21), and (5.24)–(5.26), we have

\begin{equation*}\begin{aligned}&\bigl\vert{{\mathcal{R}}^n_5[f](\hat{x}),k}\bigr\vert\\ &\mspace{20mu}\leq \widetilde{C}\widetilde{N}_0m_0\sum_{k,\ell\in{\mathscr{K}}}\vert{{\mathcal{T}}_{k\ell}}\vert\biggl(2\widetilde{C}\widetilde{N}_0m_0^2\frac{(n^{\beta}\vert{\hat{x}}\vert +n)}{n^{\alpha+\beta}}\vert{\nabla f(\hat{x})}\vert\\ &\mspace{40mu} + 2\widetilde{C}\widetilde{N}_0m_0\vert{\hat{x}}\vert\frac{(n^{\beta}\vert{\hat{x}}\vert +n)}{n^{\alpha}}\max_{z\in{\mathscr{Z}}^{\,\,n}}\vert{\nabla f(\hat{x} + n^{-\beta}z) - \nabla f(\hat{x})}\vert\\ &\mspace{40mu} + \widetilde{C}\widetilde{N}_0m_0^3\frac{(n^{\beta}\vert{\hat{x}}\vert +n)}{n^{\alpha+2\beta}} \vert{\nabla^2 f(\hat{x})}\vert\\[4pt] &\mspace{40mu} + 2\widetilde{C}\widetilde{N}_0m_0^2\frac{\bigl(n^{-\beta}\vert{\hat{x}}\vert + n^{1- 2\beta}\bigr)(n^{\beta}\vert{\hat{x}}\vert +n)}{n^{\alpha}}\max_{z\in{\mathscr{Z}}^{\,\,n}}\vert{\nabla f(\hat{x} + n^{-\beta}z) - \nabla f(\hat{x})}\vert\biggr),\end{aligned}\end{equation*}

which establishes (5.15). The estimate for ${\mathcal{R}}^n_6$ in (5.16) obtained in a similar manner by applying (2.21) and (5.22)–(5.24). This completes the proof.

We borrow the following estimates for solutions to the Poisson equation for the operator ${\mathscr{A}}^n$ from [Reference Gurvich11, Theorem 4.1] and the discussion following this theorem. Recall that $\nu^n$ is the steady-state distribution of $\hat{Y}^n$ in (2.27).

Lemma 5.3. Grant Assumption 2.2, and fix a function ${\mathscr{V}}$ in Assumption 2.3. Let $f\in C^{0,1}(\mathbb{R}^d)$ be such that $\Vert{f}\Vert_{C^{0,1}({\mathscr{B}}_x)} \leq {\mathscr{V}}(x)$ and $\nu^n(f) = 0$ . Then the function $u^n_f\in C^2(\mathbb{R}^d)$ defined by

\begin{equation*}u^n_f(x) \,:=\, \int_0^{\infty} \mathbb{E}_x\bigl[f\bigl(\widehat{Y}^n(s)\bigr)\bigr]\,\textrm{d}{s}\end{equation*}

is the unique (up to an additive constant) solution to the Poisson equation

(5.27) \begin{equation}{\mathscr{A}}^n u = -f,\end{equation}

and satisfies

(5.28) \begin{equation}\vert{\nabla u^n_f(x)}\vert \,\in\, {\mathscr{O}}\bigl((1 + \vert{x}\vert){\mathscr{V}}(x)\bigr),\qquad \bigl\vert{\nabla^2 u^n_f(x)}\bigr\vert \,\in\,{\mathscr{O}}\bigl((1 + \vert{x}\vert^2){\mathscr{V}}(x)\bigr),\end{equation}

and

(5.29) \begin{equation}\bigl[u^n_f\bigr]_{2,1;B_{\frac{m_0}{\sqrt{n}}}(x)} \,\in\,{\mathscr{O}}\bigl((1 + \vert{x}\vert^3){\mathscr{V}}(x) \bigr).\end{equation}

In the following lemma, we consider the solution of the Poisson equation in (5.27) and establish an estimate for the sum of terms ${\mathcal{R}}^n_i[u^n_f]$ , $i=1,\dotsc,6$ , given in Definition 5.1.

Lemma 5.4. Grant Assumption 2.2, and fix a function ${\mathscr{V}}$ in Assumption 2.3. Let f and $u^n_f$ be as in Lemma 5.3. Then

(5.30) \begin{equation}\sum_{j=1}^{6}{\mathcal{R}}^n_j[u^n_f](\hat{x},k) = {\mathscr{O}}\biggl(\frac{1}{n^{\alpha/2\wedge1/2}}\biggr){\mathscr{O}}\bigl((1 + \vert{\hat{x}}\vert^5){\mathscr{V}}(\hat{x})\bigr)\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}.\end{equation}

Proof. Note that

\begin{equation*}[{\mathcal{D}}^n_z]^{0}u^n_f(\hat{x}) = n^{-2\beta}\sum_{i,j\in{\mathscr{I}}}z_iz_j\partial_{ij}\bigl(u^n_f(\hat{x} + \varepsilon^n_{x,z}) - u^n_f(\hat{x})\bigr)\end{equation*}

for $\varepsilon^n_{\hat{x},z}\in\prod_{i\in{\mathscr{I}}}[\hat{x}_i, \hat{x}_i + n^{-\beta}z_i]$ . Applying (4.6) and (5.29), we obtain

(5.31) \begin{equation}{\mathcal{R}}^n_1[u^n_f](\hat{x},k) = \frac{1}{n^{\beta}}{\mathscr{O}}\bigl((1 + \vert{\hat{x}}\vert^4){\mathscr{V}}(\hat{x})\bigr)\qquad \forall\, (\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}.\end{equation}

By (5.18), we have

\begin{equation*}\vert{\bar{\Gamma}^n_{ij}(\hat{x},k) - \bar{\Gamma}^n_{ij}(0,k)}\vert \,\leq\,\widetilde{C}\widetilde{N}_0m_0^2n^{-\beta}\vert{\hat{x}}\vert\qquad \forall\, (\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation*}

and thus it follows by (5.28) that

(5.32) \begin{equation}{\mathcal{R}}^n_2[u^n_f](\hat{x}) = \frac{1}{n^{\beta}}{\mathscr{O}}\bigl((1 + \vert{\hat{x}}\vert^3){\mathscr{V}}(\hat{x})\bigr).\end{equation}

Applying Definition 5.1, (5.18), (5.23), and (5.28), we obtain

(5.33) \begin{equation}{\mathcal{R}}^n_3[u^n_f](\hat{x},k) = \frac{1}{n^{\alpha + \beta - 1}}{\mathscr{O}}\bigl((1 + \vert{\hat{x}}\vert^3){\mathscr{V}}(\hat{x})\bigr)\qquad \forall\,k\in{\mathscr{K}}.\end{equation}

Repeating the above procedure, and using Definition 5.1, (5.17), (5.23), and (5.29), we obtain

(5.34) \begin{equation}{\mathcal{R}}^n_4[u^n_f](\hat{x},k) = {\mathscr{O}}\biggl(\frac{1}{n^{\alpha+3\beta-2}}\biggr){\mathscr{O}}\bigl((1 + \vert{\hat{x}}\vert^4){\mathscr{V}}(\hat{x})\bigr) \qquad \forall\,k\in{\mathscr{K}} .\end{equation}

It follows by Lemma 5.2, (5.28), and (5.29) that

(5.35) \begin{align}{\mathcal{R}}^n_5[u^n_f](\hat{x},k) & = {\mathscr{O}}\biggl(\frac{1}{n^{\alpha + \beta- 1}}\biggr){\mathscr{O}}\bigl((1 + \vert{\hat{x}}\vert^5){\mathscr{V}}(\hat{x})\bigr) \end{align}

and

(5.36) \begin{align}{\mathcal{R}}^n_6[u^n_f](\hat{x},k) & = {\mathscr{O}}\biggl(\frac{1}{n^{2\alpha + 3\beta- 3}}\biggr){\mathscr{O}}\bigl((1 + \vert{\hat{x}}\vert^5){\mathscr{V}}(\hat{x})\bigr) \end{align}

for all $k\in{\mathscr{K}}$ . On the other hand, when $\alpha > 1$ , $\beta = \frac{1}{2}$ , $\alpha + \beta - 1 \geq \beta$ and $2\alpha + 3\beta -3 \geq \beta$ , and when $\alpha \leq 1$ , $\alpha + \beta - 1 = 2\alpha + 3\beta - 3 = \alpha/2$ and $\alpha + 3\beta - 2 = \beta$ . Then, by using (5.31)–(5.36), we have shown (5.30). This completes the proof.

Proof of Theorem 2.2. Without loss of generality, we assume that $\nu^n(f) = 0$ (see [Reference Gurvich11, Remark 3.2]). Recall the function $g^n$ in (5.6). Applying Lemma 5.1, it follows that

(5.37) \begin{equation}\begin{aligned}&\mathbb{E}_{\pi^n}\Bigl[u^n_f\bigl(\widehat{X}^n(T)\bigr)+ g^n[u^n_f]\bigl(\widehat{X}^n(T),J^n(T)\bigr)\Bigr] \\[4pt] &\mspace{60mu} = \mathbb{E}_{\pi^n}\Bigl[u^n_f(\widehat{X}^n(0))+ g^n[u^n_f](\widehat{X}^n(0),J^n(0))\Bigr] \\[4pt] &\mspace{100mu}+ \mathbb{E}_{\pi^n}\biggl[\int_0^{T}{\mathscr{A}}^n u^n_f\bigl(\widehat{X}^n(s)\bigr)\,\textrm{d}{s}\biggr]\\[4pt] &\mspace{150mu}+\sum_{j=1}^{6}\mathbb{E}_{\pi^n}\biggl[\int_0^{T}{\mathcal{R}}^n_j[u_f^n]\bigl(\widehat{X}^n(s),J^n(s)\bigr)\,\textrm{d}{s}\biggr].\end{aligned}\end{equation}

By Lemma 5.4, we have

(5.38) \begin{equation}\begin{aligned}\Biggl|\sum_{j=1}^{6}\mathbb{E}_{\pi^n}\biggl[\int_0^{T} & {\mathcal{R}}^n_j[u_f^n]\bigl(\widehat{X}^n(s),J^n(s)\bigr)\,\textrm{d}{s}\biggr] \Biggr|\\[4pt] &\,\leq\, {\mathscr{O}}\biggl(\frac{1}{n^{\alpha/2\wedge1/2}}\biggr)\mathbb{E}_{\pi^n}\biggl[\int_0^{T}\Bigl(1 + {\mathscr{V}}\bigl(\widehat{X}^n(s)\bigr)\bigl(1 + \vert{\widehat{X}^n(s)}\vert^5\bigr)\Bigr)\,\textrm{d}{s}\biggr]\\[4pt] & = {\mathscr{O}}\biggl(\frac{1}{n^{\alpha/2\wedge1/2}}\biggr)T \int_{\mathbb{R}^d\times{\mathscr{K}}}(1 +{\mathscr{V}}(\hat{x}))(1 + \vert{\hat{x}}\vert)^5\pi^n(\textrm{d}{\hat{x}},\textrm{d}{k}).\end{aligned}\end{equation}

Applying (5.6), (5.24), and (5.28), we obtain

(5.39) \begin{equation}\vert{g^n(\hat{x},k)}\vert \,\leq\, C_1\bigl(1 + (1 + \vert{\hat{x}}\vert^3){\mathscr{V}}(\hat{x})\bigr)\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation}

for some positive constant $C_1$ and all large enough n. Since $\bigl\vert{u^n_f}\bigr\vert\in{\mathscr{O}}({\mathscr{V}})$ by the claim in (22) of [Reference Gurvich11], it follows by (5.39) that

(5.40) \begin{equation}\begin{aligned}&\Bigl\vert{\mathbb{E}_{\pi^n}\Bigl[u^n_f\bigl(\widehat{X}^n(T)\bigr) + g^n[u^n_f] \bigl(\widehat{X}^n(T),J^n(T)\bigr)\Bigr]}\Bigr\vert \\[4pt] &\,\leq\,C_2\biggl( 1 + \int_{\mathbb{R}^d\times{\mathscr{K}}}{\mathscr{V}}(\hat{x})(1 + \vert{\hat{x}}\vert)^3\pi^n(\textrm{d}{\hat{x}},\textrm{d}{k})\biggr)\end{aligned}\end{equation}

for some positive constant $C_2$ . By (5.27),

(5.41) \begin{equation}\mathbb{E}_{\pi^n}\biggl[\int_0^{T}{\mathscr{A}}^n u^n_f\bigl(\widehat{X}^n(s)\bigr)\,\textrm{d}{s}\biggr] = - \mathbb{E}_{\pi^n}\biggl[\int_0^{T}f\bigl(\widehat{X}^n(s)\bigr)\,\textrm{d}{s}\biggr] = - T\pi^n(f).\end{equation}

Since $\pi^n$ is the stationary distribution, the bound in (5.40) also holds for the first term on the right-hand side of (5.37). Thus, applying (5.37), (5.38), (5.40), and (5.41), we obtain

(5.42) \begin{equation}\begin{aligned}T\bigl\vert{\pi^n(f)}\bigr\vert &\,\leq\, 2C_2\biggl( 1 + \int_{\mathbb{R}^d\times{\mathscr{K}}}{\mathscr{V}}(\hat{x})(1 + \vert{\hat{x}}\vert)^3\pi^n(\textrm{d}{\hat{x}},\textrm{d}{k})\biggr) \\[4pt] &\mspace{60mu} + {\mathscr{O}}\biggl(\frac{1}{n^{\alpha/2\wedge1/2}}\biggr)T \int_{\mathbb{R}^d\times{\mathscr{K}}}(1 +{\mathscr{V}}(\hat{x}))(1 + \vert{\hat{x}}\vert)^5\pi^n(\textrm{d}{\hat{x}},\textrm{d}{k}).\end{aligned}\end{equation}

Therefore, dividing both sides of (5.42) by T, taking $T\rightarrow\infty$ , and applying (2.32), we obtain

\begin{equation*}\vert{\pi^n(f)}\vert = {\mathscr{O}}\biggl(\frac{1}{n^{\alpha/2\wedge1/2}}\biggr).\end{equation*}

This completes the proof.

Proof of Corollary 2.3. We claim that for some positive constants $C_1$ , $\kappa_1$ , and $\kappa_2$ , a ball ${\mathscr{B}}$ , and a sequence $\epsilon_n\rightarrow 0$ as $n\rightarrow\infty$ , we have

(5.43) \begin{equation}\begin{aligned}\widehat{\mathscr{\!\!L}}^{\,\,n} \widetilde{{\mathscr{V}}}(\hat{x}) + \widehat{\mathscr{\!\!L}}^{\,\,n} g^n[\widetilde{{\mathscr{V}}}](\hat{x},k)& = {\mathscr{A}}^n \widetilde{{\mathscr{V}}}(\hat{x}) + \sum_{i=1}^6 {\mathcal{R}}^n_i[\widetilde{{\mathscr{V}}}](\hat{x},k)\\&\,\leq\, \kappa_1\mathbb{1}_{{{\mathscr{B}}}}(\hat{x}) - \kappa_2 \widetilde{{\mathscr{V}}}(\hat{x}) + C_1 +\epsilon_n\widetilde{{\mathscr{V}}}(\hat{x})\end{aligned}\end{equation}

for all $(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}}$ . Indeed the equality in (5.43) follows from Lemma 5.1. Following the calculation in the proof of Lemma 5.4, and using (2.35), the inequality in (5.43) follows from Assumption 2.3 and Lemma 5.2. By Assumption 2.2 and (2.35), we have

(5.44) \begin{equation}C_2\widetilde{{\mathscr{V}}}(\hat{x}) - C_3\,\leq\, \widetilde{{\mathscr{V}}}(\hat{x}) + g^n[\widetilde{{\mathscr{V}}}](\hat{x},k)\,\leq\, C_3(\widetilde{{\mathscr{V}}}(\hat{x}) + 1)\qquad \forall\,(\hat{x},k)\in\widehat{{\mathfrak{X}}}^n\times{\mathscr{K}},\end{equation}

for some positive constants $C_2$ and $C_3$ . Combining (5.43) and (5.44), we see that $V(\hat{x},k):= \widetilde{{\mathscr{V}}}(\hat{x}) + g^n[\widetilde{{\mathscr{V}}}](\hat{x},k)$ satisfies $\widehat{\mathscr{\!\!L}}^{\,\,n} V(\hat{x},k) \leq \kappa_3 \mathbb{1}_{{\mathscr{B}}^{\prime}}(x) -\kappa_4 V(\hat{x},k)$ for some positive constants $\kappa_3$ and $\kappa_4$ , and a ball ${\mathscr{B}}^{\prime}$ . This together with (5.44) and the hypothesis in (2.34) implies (2.32), and completes the proof.

Appendix A. The diffusion limit

Proposition A.1, which follows, shows that under suitable assumptions, the processes $\widehat{X}^n$ in (2.24) and $\widehat{Y}^n$ in (2.27) have the same diffusion limit. This proposition is interesting in its own right.

Let $(\mathbb{D}^d,{\mathcal{J}}_1)$ denote the space of $\mathbb{R}^{d}$ -valued càdlàg functions endowed with the ${\mathcal{J}}_1$ topology (see, e.g., [Reference Billingsley6]).

Proposition A.1. Grant Assumption 2.2. In addition, suppose that $\widehat{X}^n(0) \Rightarrow y_0$ ,

(A.1) \begin{equation}\frac{\xi^n_z(x^n_* + n^{\beta}\hat{x},k) - \xi^n_z(x^n_*,k)}{n^{\beta}}\,\xrightarrow[n\to\infty]{}\, \widehat{\xi}_z(\hat{x},k) \qquad \forall\,(k,z)\in{\mathscr{K}}\times{\mathscr{Z}}^{\,\,n}\end{equation}

uniformly on compact sets in $\mathbb{R}^d$ , $\widehat{M}^n$ is a square-integrable martingale, and

(A.2) \begin{equation}\frac{\Xi^n(x^n_*,k)}{n}\,\xrightarrow[n\to\infty]{}\, \overline{\Xi}(k)\,\in\,\mathbb{R}^d\qquad \forall\,k\in{\mathscr{K}}.\end{equation}

Then $\widehat{X}^n$ and $\widehat{Y}^n$ have the same diffusion limit $\widehat{X}$ in $(\mathbb{D}^d,{\mathcal{J}}_1)$ , and $\widehat{X}$ is the strong solution of the stochastic differential equation

\begin{equation*}\textrm{d} \widehat{X}(t) = \bar{b}\bigl(\widehat{X}(t)\bigr)\,\textrm{d}{t}+ \sigma_\alpha \textrm{d} W(t),\end{equation*}

with $\widehat{X}(0) = y_0$ , where

\begin{equation*}\bar{b}(\hat{x})\,:=\, \sum_{k\in{\mathscr{K}}}\pi_k\sum_{z}z\,\widehat{\xi}_z(\hat{x},k),\end{equation*}
\begin{equation*}(\sigma_\alpha)^{\mathsf{T}}\sigma_\alpha \,:=\, \begin{cases}\sum_{k\in{\mathscr{K}}}\pi_k\bar{\Gamma}(k) &\quad {for }\ \alpha>1, \\[3pt] \sum_{k\in{\mathscr{K}}}\pi_k\bar{\Gamma}(k) + \Theta &\quad {for }\ \alpha = 1, \\[3pt] \Theta &\quad {for }\ \alpha <1,\end{cases}\end{equation*}

and $\Theta = [\theta_{ij}]$ is defined by

\begin{equation*}\theta_{ij} \,:=\,2\sum_{k,\ell\in{\mathscr{K}}}\overline{\Xi}_i(k)\overline{\Xi}_j(\ell)\pi_k\Upsilon_{k\ell},\qquad i,j\in{\mathscr{I}}.\end{equation*}

Proof. Recall that $\sum_{k\in{\mathscr{K}}} \pi_k\Xi^n({x}^n_*,k) = 0$ and $\widehat{\Xi}^n(0,k) = n^{-\beta}\Xi^n({x}^n_*,k)$ . Recall the representation of $\widehat{X}^n$ in (2.24). By [Reference Pang, Talreja and Whitt21, Lemma 5.8], $\widehat{M}^n$ is stochastically bounded; see also the proof of [Reference Arapostathis, Das, Pang and Zheng3, Theorem 2.1(i)]. Since $\widehat{\Xi}^n$ is Lipschitz continuous by (2.21), it follows by the same argument as in the proof of [Reference Pang, Talreja and Whitt21, Lemma 5.5] that $\widehat{X}^n$ is stochastically bounded. Thus, by [Reference Pang, Talreja and Whitt21, Lemma 5.9], $n^{-1}X^n$ converges to the zero process in $(\mathbb{D}^d,{\mathcal{J}}_1)$ . We write $\widehat{X}^n$ as

(A.3) \begin{equation}\begin{aligned}\widehat{X}^n(t) & = \widehat{X}^n(0) + \sum_{k\in{\mathscr{K}}}\int_0^{t}\bigl(\widehat{\Xi}^n(\widehat{X}^n(s),k) - \widehat{\Xi}^n(0,k)\bigr)\mathbb{1}_k\bigl(J^n(s)\bigr)\,\textrm{d}{s} + \widehat{M}^n(t)\\&\mspace{150mu}+ \sum_{k\in{\mathscr{K}}}\frac{\Xi^n({x}^n_*,k)}{n}n^{1 - \beta}\int_0^{t}\bigl(\mathbb{1}_k\bigl(J^n(s)\bigr) - \pi_k\bigr)\,\textrm{d}{s}.\end{aligned}\end{equation}

Let $\hat{S}^n(t)$ and $\hat{R}^n(t)$ be d-dimensional processes denoting the second and fourth terms on the right-hand side of (A.3). It follows by [Reference Anderson, Blom, Mandjes, Thorsdottir and De Turck1, Proposition 3.2] and (2.23) that

(A.4) \begin{equation}\hat{R}^n \,\Rightarrow\,\begin{cases} W_R &\quad \text{for}\ \alpha\,\leq\,1, \\[4pt] 0 &\quad \text{for}\ \alpha\,>\,1,\end{cases}\quad \text{in} \quad (\mathbb{D}^d,{\mathcal{J}}_1),\end{equation}

as $n\rightarrow\infty$ , where $W_R$ is a d-dimensional Wiener process with the covariance matrix $\Theta$ . On the other hand, we have

(A.5) \begin{equation}\begin{aligned}&\hat{S}^n(t) \\& = \sum_{k\in{\mathscr{K}}}\int_0^{t}n^{-\alpha/2}\bigl(\widehat{\Xi}^n(\widehat{X}^n(s),k) - \widehat{\Xi}^n(0,k)\bigr)\,\textrm{d}\,\biggl(n^{\alpha/2}\int_0^{s}\bigl(\mathbb{1}_k(J^n(u))- \pi_k\bigr)\,\textrm{d}{u}\biggr) \\&\mspace{200mu}+ \sum_{k\in{\mathscr{K}}}\pi_k\int_0^{t}\bigl(\widehat{\Xi}^n(\widehat{X}^n(s),k)- \widehat{\Xi}^n(0,k)\bigr)\,\textrm{d}{s}.\end{aligned}\end{equation}

It follows by the convergence of $n^{-1}X^n$ to the zero process that $n^{-\alpha/2}\widehat{X}^n$ also converges to the zero process uniformly on compact sets in probability. Note that, for some constant C, we have $\vert{\widehat{\Xi}^n(\widehat{X}^n(s),k)- \widehat{\Xi}^n(0,k)}\vert \leq C\vert{\widehat{X}^n(s)}\vert$ for all $s\geq 0$ by (2.21). It then follows by [Reference Anderson, Blom, Mandjes, Thorsdottir and De Turck1, Proposition 3.2] and [Reference Jansen, Mandjes, De Turck and Wittevrongel14, Theorem 5.2] that the first term on the right-hand side of (A.5) converges to the zero process uniformly on compact sets in probability, as $n\rightarrow\infty$ . See also the proofs of Lemma 4.4 in [Reference Jansen, Mandjes, De Turck and Wittevrongel14] and Lemma 4.1 in [Reference Arapostathis, Das, Pang and Zheng3]. It is clear by (A.1) that

(A.6) \begin{equation}h^n(\hat{x}) \,:=\, \sum_{k\in{\mathscr{K}}}\pi_k\bigl(\widehat{\Xi}^n(\hat{x},k)- \widehat{\Xi}^n(0,k)\bigr) \,\longrightarrow\,\sum_{k\in{\mathscr{K}}}\pi_k\sum_{z}z\,\widehat{\xi}_z(\hat{x},k)\end{equation}

uniformly on compact sets in $\mathbb{R}^d$ . Note that the function $h^n$ is Lipschitz continuous by (2.21). By [Reference Pang, Talreja and Whitt21, Theorem 4.1] (see also [Reference Jansen, Mandjes, De Turck and Wittevrongel14, Lemma 4.1]), the integral mapping $x^n = \Psi^n(z^n): \mathbb{D}^d\rightarrow\mathbb{D}^d$ defined by

\begin{equation*}x^n(t) = z^n(t) + \int_0^{t} h^n(x^n(s)) \,\textrm{d}{s}\qquad \forall\,n\in\mathbb{N},\end{equation*}

is continuous in $(\mathbb{D}^d,{\mathcal{J}}_1)$ . Thus, applying the continuous mapping theorem and using (A.3)–(A.6), we obtain

\begin{equation*}\widehat{X}^n \,\Rightarrow\, \widehat{X} \quad \text{in} \quad (\mathbb{D}^d,{\mathcal{J}}_1).\end{equation*}

Recall the definitions of $\bar{\Gamma}^n$ and $\Theta^n$ in (2.25) and (2.29), respectively. As $n\rightarrow\infty$ , we have that $\bar{\Gamma}^n(0,k)\rightarrow\bar{\Gamma}(k)$ when $\alpha\geq 1$ , and $\bar{\Gamma}^n(0,k) \rightarrow 0$ when $\alpha < 1$ , by (2.23). Since $\beta = \max\{1-\alpha/2,1/2\}$ , it then follows by (A.2) that $\Theta^n \rightarrow \Theta$ when $\alpha \leq 1$ , and $\Theta^n \rightarrow 0$ when $\alpha > 1$ . It is then straightforward to verify that $\widehat{Y}^n\Rightarrow\widehat{X}$ in $(\mathbb{D}^d,{\mathcal{J}}_1)$ , as $n\rightarrow\infty$ . Therefore, $\widehat{X}^n$ and $\widehat{Y}^n$ have the same diffusion limit.

Acknowledgements

This research was supported in part by the Army Research Office through Grant W911NF-17-1-001, and in part by the National Science Foundation through Grants CMMI-1635410, DMS-1715210, and DMS-1715875.

References

Anderson, D., Blom, J., Mandjes, M., Thorsdottir, H. and De Turck, K. (2016). A functional central limit theorem for a Markov-modulated infinite-server queue. Methodology Comput. Appl. Prob. 18, 153168.CrossRefGoogle Scholar
Arapostathis, A., Biswas, A. and Pang, G. (2015). Ergodic control of multi-class $M/M/N+M$ queues in the Halfin–Whitt regime. Ann. Appl. Prob. 25, 35113570.CrossRefGoogle Scholar
Arapostathis, A., Das, A., Pang, G. and Zheng, Y. (2019). Optimal control of Markov-modulated multiclass many-server queues. Stoch. Systems 9, 155181.CrossRefGoogle Scholar
Arapostathis, A., Hmedi, H. and Pang, G. (2020). On uniform exponential ergodicity of Markovian multiclass many-server queues in the Halfin–Whitt regime. To appear in Math. Operat. Res. Google Scholar
Arapostathis, A., Pang, G. and Sandrić, N. (2019). Ergodicity of a Lévy-driven SDE arising from multiclass many-server queues. Ann. Appl. Prob. 29, 10701126.CrossRefGoogle Scholar
Billingsley, P. (1999). Convergence of Probability Measures, 2nd edn. John Wiley, New York.CrossRefGoogle Scholar
Coolen-Schrijner, P. and van Doorn, E. A. (2002). The deviation matrix of a continuous-time Markov chain. Prob. Eng. Inf. Sci. 16, 351366.10.1017/S0269964802163066CrossRefGoogle Scholar
Dai, J. G., He, S. and Tezcan, T. (2010). Many-server diffusion limits for $G/PH/n+GI$ queues. Ann. Appl. Prob. 20, 18541890.CrossRefGoogle Scholar
Dieker, A. B. and Gao, X. (2013). Positive recurrence of piecewise Ornstein–Uhlenbeck processes and common quadratic Lyapunov functions. Ann. Appl. Prob. 23, 12911317.10.1214/12-AAP870CrossRefGoogle Scholar
Fralix, B. H. and Adan, I. J. B. F. (2009). An infinite-server queue influenced by a semi-Markovian environment. Queueing Systems 61, 6584.CrossRefGoogle Scholar
Gurvich, I. (2014). Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. Ann. Appl. Prob. 24, 25272559.CrossRefGoogle Scholar
Hmedi, H., Arapostathis, A. and Pang, G. (2019). Uniform stability of a class of large-scale parallel server networks. Preprint. Available at https://arxiv.org/abs/1907.04793.Google Scholar
Hunter, J. J. (1982). Generalized inverses and their application to applied probability problems. Linear Algebra Appl. 45, 157198.CrossRefGoogle Scholar
Jansen, H. M., Mandjes, M., De Turck, K. and Wittevrongel, S. (2019). Diffusion limits for networks of Markov-modulated infinite-server queues. Performance Evaluation 135.CrossRefGoogle Scholar
Jonckheere, M. and Shneer, S. (2014). Stability of multi-dimensional birth-and-death processes with state-dependent 0-homogeneous jumps. Adv. Appl. Prob. 46, 5975.CrossRefGoogle Scholar
Khasminskii, R. Z. (2012). Stability of regime-switching stochastic differential equations. Probl. Inf. Transm. 48, 259270.CrossRefGoogle Scholar
Khasminskii, R. Z., Zhu, C. and Yin, G. (2007). Stability of regime-switching diffusions. Stoch. Process. Appl. 117, 10371051.CrossRefGoogle Scholar
Kumar, R., Lewis, M. E. and Topaloglu, H. (2013). Dynamic service rate control for a single-server queue with Markov-modulated arrivals. Naval Res. Logistics 60, 661677.10.1002/nav.21560CrossRefGoogle Scholar
Mao, X. (1999). Stability of stochastic differential equations with Markovian switching. Stoch. Process. Appl. 79, 4567.CrossRefGoogle Scholar
Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. Appl. Prob. 25, 518548.CrossRefGoogle Scholar
Pang, G., Talreja, R. and Whitt, W. (2007). Martingale proofs of many-server heavy-traffic limits for Markovian queues. Prob. Surveys 4, 193267.CrossRefGoogle Scholar
Pang, G. and Zheng, Y. (2017). On the functional and local limit theorems for Markov modulated compound Poisson processes. Statist. Prob. Lett. 129, 131140.CrossRefGoogle Scholar
Shao, J. and Xi, F. (2014). Stability and recurrence of regime-switching diffusion processes. SIAM J. Control Optimization 52, 34963516.CrossRefGoogle Scholar
Xia, L., He, Q. and Alfa, A. S. (2017). Optimal control of state-dependent service rates in a MAP/M/1 queue. IEEE Trans. Automatic Control 62, 49654979.CrossRefGoogle Scholar
Zeifman, A. I. (1998). Stability of birth-and-death processes. J. Math. Sci. 91, 30233031.CrossRefGoogle Scholar
Zhu, Y. and Prabhu, N. U. (1991). Markov-modulated $PH/G/1$ queueing systems. Queueing Systems 9, 313322.CrossRefGoogle Scholar