Let $q\ge2$ be an integer, $\{X_n\}_{n\geq 1}$ a stochastic process with state space $\{0,\ldots,q-1\}$ , and F the cumulative distribution function (CDF) of $\sum_{n=1}^\infty X_n q^{-n}$ . We show that stationarity of $\{X_n\}_{n\geq 1}$ is equivalent to a functional equation obeyed by F, and use this to characterize the characteristic function of X and the structure of F in terms of its Lebesgue decomposition. More precisely, while the absolutely continuous component of F can only be the uniform distribution on the unit interval, its discrete component can only be a countable convex combination of certain explicitly computable CDFs for probability distributions with finite support. We also show that $\mathrm{d} F$ is a Rajchman measure if and only if F is the uniform CDF on [0, 1].

A generalization of Downton's (1968) carrier-borne epidemic process is considered in which the assumption of homogeneous mixing is replaced by a general infection mechanism. Using a method and notation similar to that of Gani and Purdue (1984), a recursive algorithm for the joint distribution of the total size and the total observed size of the epidemic is obtained.

The use of Mineka coupling is extended to a case with a continuous state space: an efficient coupling of random walks S and S' in can be made such that S' — S is virtually a one-dimensional simple random walk. This insight settles a zero-two law of ergodicity. One more proof of Blackwell's renewal theorem is also presented.

In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

A random vibration model is investigated in this paper. The model is formulated as a cosine function with a constant frequency and a random walk phase. We show that this model is second-order stationary and can be rewritten as a vector-valued AR(1) model as well as a scalar ARMA(2, 1) model. The linear innovation sequence of the AR(1) model is shown to be a martingale difference sequence while the linear innovation sequence of the ARMA(2, 1) model is only an uncorrelated sequence. A non-linear predictor is derived from the AR(1) model while a linear predictor is derived from the ARMA(2, 1) model. We deduce that the non-linear predictor of this model has less mean square error than that of the linear predictor. This has significance, for example, for predicting seasonal phenomena with this model. In addition, the limit distributions of the sample mean, the finite Fourier transforms and the autocovariance functions are derived using a martingale approach. The limit distribution of autocovariance functions differs from the classical result given by Bartlett's formula.

This paper presents new methods for selecting the best population value under the classical secretary problem sequential selection rules. Unlike the classical case, the possible population arrival sequences considered here are not necessarily equally likely. When arrival orders in the sequence are rank dependent, the performance of the optimal solution to the standard problem (totally random arrival order assumed) is very poor, compared with procedures designed to guard against non-uniform arrival orders.

We present a stand-alone simple proof of a probabilistic interpretation of the Gaussian binomial coefficients by conditioning a random walk to hit a given lattice point at a given time.

We pose an optimal control problem arising in a perhaps new model for retirement investing. Given a control function f and our current net worth X(t) for any t, we invest an amount f(X(t)) in the market. We need a fortune of M ‘superdollars’ to retire and want to retire as early as possible. We model our change in net worth over each infinitesimal time interval by the Itô process dX(t) = (1 + f(X(t)))dt + f(X(t))dW(t). We show how to choose the optimal f = f0 and show that the choice of f0 is optimal among all nonanticipative investment strategies, not just among Markovian ones.

Consider n cells, of which some are target cells, and suppose that each cell has a weight. The cells are killed in a sequential manner, with each currently live cell being the next one killed with a probability proportional to its weight. We study the distribution of the number of cells that are alive at the moment when all the target cells have been killed.

The aim of this paper is to provide the conditions necessary to reduce the complexity of state filtering for finite stochastic systems (FSSs). A concept of lumpability for FSSs is introduced. In this paper we assert that the unnormalised filter for a lumped FSS has linear dynamics. Two sufficient conditions for such a lumpability property to hold are discussed. We show that the first condition is also necessary for the lumped FSS to have linear dynamics. Next, we prove that the second condition allows the filter of the original FSS to be obtained directly from the filter for the lumped FSS. Finally, we generalise an earlier published result for the approximation of a general FSS by a lumpable FSS.

Let {Zn}n≥0 be a random walk with a negative drift and independent and identically distributed increments with heavy-tailed distribution, and let M = supn≥0Zn be its supremum. Asmussen and Klüppelberg (1996) considered the behavior of the random walk given that M > x for large x, and obtained a limit theorem, as x → ∞, for the distribution of the quadruple that includes the time τ = τ(x) to exceed level x, position Zτ at this time, position Zτ-1 at the prior time, and the trajectory up to it (similar results were obtained for the Cramér-Lundberg insurance risk process). We obtain here several extensions of this result to various regenerative-type models and, in particular, to the case of a random walk with dependent increments. Particular attention is given to describing the limiting conditional behavior of τ. The class of models includes Markov-modulated models as particular cases. We also study fluid models, the Björk-Grandell risk process, give examples where the order of τ is genuinely different from the random walk case, and discuss which growth rates are possible. Our proofs are purely probabilistic and are based on results and ideas from Asmussen, Schmidli and Schmidt (1999), Foss and Zachary (2002), and Foss, Konstantopoulos and Zachary (2007).

In this paper we refine a Poisson limit theorem of Gnedenko and Kolmogorov (1954): we determine the error order of a Poisson approximation for sums of asymptotically negligible integer-valued random variables that converge in distribution to the Poisson law. As an application of our results, we investigate the case of the coupon collector's problem when the distribution of the collector's waiting time is asymptotically Poisson.

Let T be a stopping time associated with a sequence of independent and identically distributed or exchangeable random variables taking values in {0, 1, 2, …, m}, and let ST,i be the stopped sum denoting the number of appearances of outcome 'i' in X1, …, XT, 0 ≤ i ≤ m. In this paper we present results revealing that, if the distribution of T is known, then we can also derive the joint distribution of (T, ST,0, ST,1, …, ST,m). Two applications, which have independent interest, are offered to illustrate the applicability and the usefulness of the main results.

We are concerned with the variation of the supercritical nearest-neighbours contact process such that first infection occurs at a lower rate; it is known that the process survives with positive probability. Regarding the rightmost infected of the process started from one site infected and conditioned to survive, we specify a sequence of space-time points at which its behaviour regenerates and, thus, obtain the corresponding strong law and central limit theorem. We also extend complete convergence in this case.

Let ξ = (ξ1, . . ., ξm) be a negatively associated mean-zero random vector with components that obey the bound |ξi| ≤ B, i = 1, . . ., m, and whose sum W = ∑i=1mξi has variance 1. The bound d1(ℒ(W), ℒ(Z)) ≤ 5B - 5.2∑i≠ jσij is obtained, where Z has the standard normal distribution and d1(∙, ∙) is the L1 metric. The result is extended to the multidimensional case with the L1 metric replaced by a smooth functions metric. Applications to second-order stationary random fields with exponential decreasing covariance are also presented.

It is shown that if ϕ is a given function out of a large class satisfying a certain regularity condition, then a supercritical age-dependent branching process {Z(t)} exists with deterministic immigration and given life-length and family-size distributions such that Z(t)/(eat ϕ(t)) converges in probability to a non-zero constant, a being the appropriate Malthusian parameter.

As an easy corollary one discovers the asymptotic behaviour of some processes with random immigration.

This paper considers a competing risks system with p pieces of software where each piece follows the model by Littlewood (1980) described as follows. The failure rate of a piece of software relies on the residual number of bugs remaining in the software where each bug produces failures at varying rates. In effect, bugs with higher failure rates tend to be observed earlier in the testing period. Tasks are assigned to the system and the task completion times as well as the software failure times are assumed to be independent of each other. The system is observed over a fixed testing period and the system reliability upon test termination is examined. An estimator of the system reliability is presented and its asymptotic properties as well as finite-sample properties are obtained.

Suppose that π is a policy for resource allocation in a stochastic environment and π ∗ is an optimal policy. Two existing procedures for policy evaluation are described and compared. Both of these evaluate π by means of upper bounds on R(π ∗) – R(π), the total reward lost when making resource allocations according to π rather than π∗. The bounds developed by these two methods are called Type 1 and Type 2. We demonstrate by example that neither of these procedures dominates the other in the sense of always yielding tighter bounds. A modification to Type 2 bounds is proposed resulting in an improved procedure which always dominates the Type 1 approach.

For a zero-mean, unit-variance stationary univariate Gaussian process we derive the probability that a record at the time n, say $X_n$ , takes place, and derive its distribution function. We study the joint distribution of the arrival time process of records and the distribution of the increments between records. We compute the expected number of records. We also consider two consecutive and non-consecutive records, one at time j and one at time n, and we derive the probability that the joint records $(X_j,X_n)$ occur, as well as their distribution function. The probability that the records $X_n$ and $(X_j,X_n)$ take place and the arrival time of the nth record are independent of the marginal distribution function, provided that it is continuous. These results actually hold for a strictly stationary process with Gaussian copulas.

We consider the two-dimensional process {X(t), V(t)} where {V(t)} is Brownian motion with drift, and {X(t)} is its integral. In this note we derive the joint density function of T and V(T) where T is the time at which the process {X(t)} first returns to its initial value. A series expansion of the marginal density of T is given in the zero-drift case. When V(0) and the drift are both positive there is a positive probability that {Χ (t)} never returns to its initial value. We show how this probability grows for small drift. Finally, using the Kontorovich–Lebedev transform pair we obtain the escape probability explicitly for arbitrary values of the drift parameter.