For every hth member of a two-state Markov chain the value of a random variable Y is observed where the distribution of Y is conditional on the state of the corresponding member of the chain. A recursive set of equations is derived giving the posterior probabilities for both the observed and unobserved members. The use of this recursive solution to investigate the optimality of certain simple classification rules is discussed, and a “classification by runs” is also presented.

This paper is a generalization to Markov chains of the work of Shepp [6] in the i.i.d case. Shepp studies the limiting values of the averages Tn = (Sn+ f(n) – Sn)/f(n) where Sn = X0 + X1+ · ·· + Xn, X0 = 0, n = 1, 2, ···, is a sum of mutually independent and identically distributed random variables. The function f takes positive integer values and non-decreasingly tends to infinity. Here we take a class of functions f in central position f(n) = [c log n], c > 0, n = 1, 2, ···. There are many refinements of the function f in the i.i.d case [1], [2]. Here we consider the more general case where X1, · ··, Xn is an irreducible and recurrent Markov chain. The state space of the chain is either compact or countable.

The mean time to extinction of a critical Galton-Watson process with initial population size k is shown to be asymptotically equivalent to two integral transforms: one involving the kth iterate of the probability generating function and one involving the generating function itself. Relating the growth of these transforms to the regular variation of their arguments, immediately connects statements involving the regular variation of the probability generating function, its iterates at 0, the quasistationary measures, their partial sums, and the limiting distribution of the time to extinction. In the critical case of finite variance we also give the growth of the mean time to extinction, conditioned on extinction occurring by time n.

In his well-known textbook Probability with Martingales, David Williams (1991) introduces the Mabinogion sheep problem in which there is a magical flock of sheep, some black, some white. At each stage n=1,2,..., a sheep (chosen randomly from the entire flock, independently of previous events) bleats; if this bleating sheep is white, one black sheep (if any remain) instantly becomes white; if the bleating sheep is black, one white sheep (if any remain) instantly becomes black. No births or deaths occur. Suppose that one may remove any number of white sheep from the flock at (the end of) each stage n=0,1,.... The object is to maximize the expected final number of black sheep. By applying the martingale optimality principle, Williams showed that the problem is solvable and admits a simple nice solution. In this paper we consider a generalization of the Mabinogion sheep problem with two parameters 0≤p, q≤1, denoted M(p,q), in which at each stage, when the bleating sheep is white (black, respectively), a black (white, respectively) sheep (if any remain) instantly becomes white (black, respectively), with probability p (q, respectively) and nothing changes with probability 1-p (1-q, respectively). Note that the original problem corresponds to (p,q)=(1,1). Following Williams' approach, we solve the two cases (p,q)=(1,1/2) and (1/2,1) which admit simple solutions.

In this paper we study the numerical approximation of the optimal long-run average cost of a continuous-time Markov decision process, with Borel state and action spaces, and with bounded transition and reward rates. Our approach uses a suitable discretization of the state and action spaces to approximate the original control model. The approximation error for the optimal average reward is then bounded by a linear combination of coefficients related to the discretization of the state and action spaces, namely, the Wasserstein distance between an underlying probability measure μ and a measure with finite support, and the Hausdorff distance between the original and the discretized actions sets. When approximating μ with its empirical probability measure we obtain convergence in probability at an exponential rate. An application to a queueing system is presented.

Continuous-time threshold autoregressive (CTAR) processes have been developed in the past few years for modelling non-linear time series observed at irregular intervals. Several approximating processes are given here which are useful for simulation and inference. Each of the approximating processes implicitly defines conditions on the thresholds, thus providing greater understanding of the way in which boundary conditions arise.

It is shown here that questions about the probability distributions of the partial sums of a sequence of geometric distributions, all with different parameters, can be answered by considering the transition probabilities of a homogeneous Markov chain. The result is applied to the embedded random walk of an epidemic process.

In this paper we present a set of results relating to the occupation time α(t) of a process X(·). The first set of results concerns exact characterizations of α(t), e.g. in terms of its transform up to an exponentially distributed epoch. In addition, we establish a central limit theorem (entailing that a centered and normalized version of α(t)∕t converges to a zero-mean normal random variable as t→∞) and the tail asymptotics of ℙ(α(t)∕t≥q). We apply our findings to spectrally positive Lévy processes reflected at the infimum and establish various new occupation time results for the corresponding model.

Consider a device which begins operating at time t = 0 and then operates continuously in time. The operation of the device is satisfactory so long as a physical parameter X(t) of the device remains less than a critical value c > 0 (X(0) = 0). The device does not operate satisfactorily (i.e., it is “failed”) at a time t if X(t) ≧ c. Suppose that X(t) is a random variable with mean EX(t) = ta (a> 0), i.e., the device parameter X drifts with a linear rate a.

A transformation is introduced to stabilize the variance of the largest value in a sample from a normal distribution. The transformed distribution is found to approximate closely to the limiting Gumbel form for all sample sizes. The practical implications of the result are discussed.

We have found a mistake in the proofs of Navarro (2008, Theorem 2.3(b) and 2.3(c)) due to misapplication of properties of hazard rate and likelihood ratio orders. In this paper we show with an example that the stated results do not hold. This example is interesting since it proves some unexpected properties for these orderings under the formation of coherent systems. The result stated in Navarro (2008, Theorem 2.3(a)) for the usual stochastic order is correct.

Consider a spectrally negative risk process where, on ruin, the deficit is immediately paid, and the process restarts from 0. When the process reaches a threshold b, all the surplus above b is paid as dividend. Applying the theory of exit times for a spectrally negative Lévy process and its reflection at the maximum and at the minimum, we obtain recursive formulae for the following moments. (i) The moments of the discounted loss until the process reaches b. This is equivalent to the moments of the discounted dividends in the dual model under the barrier strategy. (ii) The moments of the discounted loss for models with and without a dividend barrier for the infinite horizon. (iii) The moments of the discounted dividends for the infinite horizon.

Distributions of some random variables have been characterized by independence of certain functions of these random variables. For example, let X and Y be two independent and identically distributed random variables having the gamma distribution. Laha showed that U = X + Y and V = X | Y are also independent random variables. Lukacs showed that U and V are independently distributed if, and only if, X and Y have the gamma distribution. Ferguson characterized the exponential distribution in terms of the independence of X – Y and min (X, Y). The best-known of these characterizations is that first proved by Kac which states that if random variables X and Y are independent, then X + Y and X – Y are independent if, and only if, X and Y are jointly Gaussian with the same variance. In this paper, Kac's hypotheses have been somewhat modified. In so doing, we obtain a larger class of distributions which we shall call class λ1. A subclass λ0 of λ1 enjoys many nice properties of the Gaussian distribution, in particular, in non-linear filtering.

Consider the optimal control problem of leaving an interval (– a, a) in a limited playing time. In the discrete-time problem, a is a positive integer and the player's position is given by a simple random walk on the integers with initial position x. At each time instant, the player chooses a coin from a control set where the probability of returning heads depends on the current position and the remaining amount of playing time, and the player is betting a unit value on the toss of the coin: heads returning +1 and tails − 1. We discuss the optimal strategy for this discrete-time game. In the continuous-time problem the player chooses infinitesimal mean and infinitesimal variance parameters from a control set which may depend upon the player's position. The problem is to find optimal mean and variance parameters that maximize the probability of leaving the interval [— a, a] within a finite time T > 0.

In this paper, we describe a class of Wiener functionals that are ‘indeterminate by their moments’, that is, whose distributions are not uniquely determined by their moments. In particular, it is proved that the integral of a geometric Brownian motion is indeterminate by its moments and, moreover, shown that previous proofs of this result are incorrect. The main result of this paper is based on geometric inequalities in Gauss space and on a generalization of the Krein criterion due to H. L. Pedersen.

In this paper we give a probabilistic interpretation for the function d(f), which has been proposed as a measure of fuzziness of a fuzzy set f. For this we construct random patterns which approximate the fuzzy set f and show that, for large n, the number of possible outcomes of the nth random pattern is approximately 2nd(f). We also give the best possible constant K in a more accurate approximation .

We construct random fields with Pólya-type autocorrelation function and dampened Pólya cross-correlation function. The marginal distribution of the random fields may be taken as any infinitely divisible distribution with finite variance, and the random fields are fully characterized in terms of their joint characteristic function. This makes available a new class of non-Gaussian random fields with flexible correlation structure for use in modeling and estimation.

Signatures are useful in analyzing and evaluating coherent systems. However, their computation is a challenging problem, especially for complex coherent structures. In most cases the reliability of a binary coherent system can be linked to a tail probability associated with a properly defined waiting time random variable in a sequence of binary trials. In this paper we present a method for computing the minimal signature of a binary coherent system. Our method is based on matrix-geometric distributions. First, a proper matrix-geometric random variable corresponding to the system structure is found. Second, its probability generating function is obtained. Finally, the companion representation for the distribution of matrix-geometric distribution is used to obtain a matrix-based expression for the minimal signature of the coherent system. The results are also extended to a system with two types of components.