1. INTRODUCTION
Mixing is now systematically used in time series where martingale
assumptions and results cannot be directly employed. Mixing has proved
particularly useful in cases where nonlinearities appear, such as
autoregressive conditional heteroskedasticity (ARCH) modeling in
econometrics. This success relies on powerful limit theorems proved
under mixing conditions (see, among others, Doukhan,
1994; Rio, 2000; Doukhan, 2002). These limit results serve as basic
tools for computation of the significance level and power of
statistical tests. Mixing assumptions can be used in more general
frameworks involving fading memory (asymptotic independence between
functions of the past and the future of the process), such as near
epoch dependence (NED) of a mixing process.
We recall here the mixing properties of some models used in
econometrics. Simultaneously, we present a different approach to limit
theorems when mixing does not hold (which really may occur, as shown in
Andrews, 1984, example (4.16)). For the sake
of simplicity, our exposition mainly focuses on one-dimensional time
series.
The paper is organized as follows. To provide deep econometric
motivations, Section 2 exposes several situations where the various
weak dependence conditions arise, and after some generic examples, we
consider specific problems, including unit root problems, parametric
problems, sieve bootstrap, and semiparametric estimation problems in
Sections 2.1, 2.2, 2.4, and 2.5. Section 2.3 considers generalized
method of moments (GMM) estimation in which the Doukhan and Louhichi
(1999) weak dependence condition allows one
to provide a complete proof of the results in Hall and Horowitz (1996). Indeed, the latter authors improperly claim
a mixing property of their models to prove their consistency results.
Finally, Section 2.6 considers nonparametric estimation problems.
Section 3 makes precise the mathematical framework of weak dependence
needed in the previous section. It describes some classical concepts of
fading memory (mixing conditions, the association property) and also
the new weak dependence conditions introduced by Doukhan and Louhichi
(1999). After this, Section 4 provides
numerous classes of models commonly used in econometrics and finance
and focuses on their weak dependence properties. Section 5 recalls some
probabilistic limit theorems available in those cases. Extensions of
Donsker's functional central limit theorem (FCLT) and the FCLT for
the cumulative distribution function are discussed. Section 6 is
devoted to functional estimation. Consistency and CLTs are discussed
here. Proofs are given in Appendix A, and Appendix B recalls the main
probabilistic tools.
Finally, we remark that the limit theorems of Section 3 and the
asymptotic results for functional estimation in Section 6 are provided
for very large classes of models (Sections 3 and 4). Hence, more
general time series formulations such as those in Section 4 allow us to
extend directly the classical results of Section 2.
2. ECONOMETRICS AND DEPENDENCE
Time series analysis is a major part of econometrics. Here we provide
several examples of interest in which it is essential to consider
dependent structures instead of simple independence. In some
situations, classical tools of weak dependence such as mixing are
useless. For instance, when bootstrap techniques are used, no mixing
conditions can be expected. Consider the following example concerning
bootstrap: let a stationary autoregressive sequence be generated by an
independent and identically distributed (i.i.d.) sequence
:
Standard nonparametric estimation techniques provide an estimate of
the autoregression function r. Let
be a convenient estimator of r. Given data
(X1,…,Xn) from
the sequence (2.1), another autoregressive process is defined by
The innovations
are drawn according to the empirical measure of the estimated residuals,
.
No mixing assumption can be expected for the previous model (2.2): see
(4.16) in Section 4. However, a new concept of fading memory can still be
applied. Bickel and Bühlmann (1999) set up
such a new weak dependence condition to build critical bootstrap values for
a linearity test in linear models. Doukhan and Louhichi (1999) have extended it to fit models such as positively
dependent sequences, Markov chains (with or without topological assumptions),
and Bernoulli shifts (see Definition 4.1). The Bernoulli shifts are defined
in Assumption 1 of Hall and Horowitz (1996) and
are used throughout that paper. The previously mentioned weak
dependence conditions yield standard results concerning convergence in
distribution with an
-normalization.
Another application of these results concerns linearity tests in time
series analysis. Rios (1996) considers a
stationary functional autoregressive model (2.1) where r =
L + C is the decomposition of the autoregression
function into a sum of linear (L) and nonlinear (C) components. Local
linearity of r is then tested via the null hypothesis
where the weight function w has compact support.
Still another problem of interest is to test the independence of the
innovations
in a regression model
This can be performed using the Durbin–Watson statistic. The
latter can be written as a continuous functional of the Donsker line of
the sequence
.
2.1. Unit Root Tests
Consider a stationary autoregressive sequence
generated by an i.i.d. sequence
,
A classical problem is to test whether there is a unit root (i.e.,
a = 1). In the specific context of aggregate time series, the
assumption of white noise innovations seems to be rather strong.
Phillips (1987) develops unit root tests for
mixing and heterogeneously distributed innovations. The ordinary least
squares estimate
is shown to be a continuous functional of the Donsker line of the sequence
.
As an application of the FCLT, Phillips shows that a unit root test can be
based on the fact that under the null hypothesis H0
: a = 1,
where W denotes the standard Brownian motion and
.
The author works with stationary strong mixing sequences, and conditions
under which the FCLT result holds true are reported in Section 5.1. This
result can be obtained under a weak dependent context detailed in Section 4.
The conditions for which Donsker's theorems hold are described in
Section 5.1. This example, as the author suggests, can be generalized
to error sequences
that allow for heteroskedasticity. See also Mills (1999) for a discussion of the Dickey–Fuller
unit root test in autoregressive models when errors fluctuate about a
nonzero mean.
2.2. Parametric Problems
GMM estimation procedures involve an estimate
,
which is a solution of the arg-min problem
,
where
Here
is a finite-dimensional parameter set, and g(·,·) is
a given function such that
,
where θ0 is the true parameter point. In the time series
context, the positive semidefinite matrix Ω is often replaced (see
Hall and Horowitz, 1996, equation (3.2)) by
an asymptotically optimal weight matrix estimate
and κ is such that
.
The statistic to test
,
where
(the square root of a symmetric positive matrix is uniquely defined).
2.2.1. Block Bootstrap.
A bootstrap procedure allows one to estimate the limit distribution
of an estimate.
We describe a block-bootstrap procedure that is adapted to the times
series
.
Let b = b(n) and l =
l(n) denote the number and the length of the blocks.
Then bl = n and the block j is
(see Künsch, 1989). In this construction, a
suitable form of
if the process Xk =
H(ξk,ξk−1,…)
is a Bernoulli shift as defined in Section 4.3. Here
are obtained through filtering and estimation procedures in the simple case
of a linear process
(H(z0,z1,…) =
[sum ]k ak
zk; see Section 2.4); in the general
setting, one needs to develop additional estimation procedures. To
describe the asymptotic properties of such processes one needs to know
the limiting asymptotic behavior of Bernoulli shifts.
2.2.2. Conditional Bootstrap.
A simple local conditional bootstrap is investigated by Ango Nze,
Bühlmann, and Doukhan (2002). In that
paper, it is shown that asymptotic properties can be obtained using the
same weak dependence techniques. The following central limit theorem
(CLT) holds under standard mixing assumptions:
where the diagonal matrix Σn has d
entries. GMM techniques naturally involve an unknown covariance matrix.
To estimate such limiting distributions it is natural to use bootstrap
techniques.
2.3. Bootstrapping Critical Values for GMM Estimators
Let
denote a block-bootstrap sample and let
.
The expectation is taken with respect to the bootstrap distribution.
The GMM estimate
solves the arg-min problem
if the matrix Ω is known.
Hall and Horowitz (1996) make the erroneous
statement that such Bernoulli shifts are strong mixing. However, the
procedure used by Hall and Horowitz makes the bootstrap work. The weak
dependence condition as defined in Doukhan and Louhichi (1999) allows us to rigorously prove the consistency
of the Hall and Horowitz procedures. More precisely, if
Xn =
h(εn,εn−1,…)
for some i.i.d. sequence
,
their Assumption 1 is
1 The function h
takes its values in Lx × 1 space equipped
with a norm ∥·∥. The assumption is missing the symbol of
mathematical expectation
,
as in Andrews (2002).
This condition holds for linear processes, and it is claimed to imply
geometric strong mixing by the authors. Simple example (4.16) proves
that this does not hold in general. This condition, however, does imply
weak dependence in Doukhan and Louhichi (1999). Consequently, a tail inequality for sums of
functions of the sequence ξn = f
(Xn,θ) can be derived. It is the main
tool to prove the validity of the bootstrap in this dependent
setting.
The preceding procedure can be used for testing the null hypothesis
H0 : θ = θ0 against the
bilateral alternative. The studentized statistic
Tn(θ) described in (2.4) and the
critical value Qα then satisfy the relation
that, under H0,
Hall and Horowitz show that the bootstrap studentized statistic
and the previously mentioned statistic Tn(θ)
have close laws, in the sense that
for a relevant integer 2a, with a ≥ 1 + ξ,
and the range of ξ ∈ [0,1] is formulated according
to the dependence assumptions prescribed. This relation comes from an
Edgeworth development. It yields an improved acceptance rule for the
test of H0:
Andrews (2002) points out that a direct
computation of the bootstrap critical value Qα*
is a hard problem and that the common estimating procedure, which is
based upon B bootstrapped, independent copies (from the law of
large numbers, it follows that
)
is also difficult to implement. Indeed, the computations involve the
minimization of B nonlinear functionals. A numerical improvement
is brought to bear in Andrews's paper. A bootstrap estimator
is computed by applying the Newton–Raphson algorithm. The initial
value is
,
and k iterations are made (k ≥ 3). A bootstrap
studentized statistic Tn,k* is now
available, for which the computation of the critical bootstrap values
Qα*(B) is much easier, because the
problem is linear. The method is claimed to be as accurate as the one
discussed by Hall and Horowitz. In fact, the author states a similar
result to (2.6) with respect to
.
The assumptions are those of Hall and Horowitz. Therefore, the previously
mentioned Assumption 1 must also be read in the context of the comments we
have already made about Hall and Horowitz's paper. For the sake of
completeness we present a corrected version of Lemma 1 in Hall and Horowitz
(1996), which is proved in Section B.4 of Appendix
B.
LEMMA 2.1. Let (ξn) be a
stationary ψ1-weakly dependent (see Definition 3.4)
sequence with
such that
for some a > 0, as r ↑ ∞, and
.
Then
2 Here θr
is a dependence coefficient, and it is not related to a statistical
parameter denoted θ and estimated by
.
satisfies
Following precisely the same steps as in Hall and Horowitz (1996), we thus prove, by only replacing their Lemma
1 by Lemma 2.1, that bootstrapping critical values for GMM estimators
are valid.
Now Theorems 1–3 in Hall and Horowitz (1996) are rigorously proved. A paper on this topic
by the authors is in preparation to provide sharper results; for
instance, the exponent 33 in the previous lemma is unnatural, and it
can be changed.
2.4. Sieve Bootstrap
Bickel and Bühlmann (1999) tackle the
problem of the “sieve bootstrap” for a one-sided linear
process
where (ξn) is a sequence of i.i.d. random
variables (r.v.s) with
and the density function fξ, and where
.
Under the assumption that the function
has no root in the closed unit circle, the process (2.7) admits an AR(∞)
representation
with
.
Equation (2.8) is fitted with an autoregressive process of finite order
p(n) (p(n)/n → 0,
p(n) → ∞). Using estimated residuals, the
resampling (i.i.d.) innovation process
is constructed by smoothing the empirical process based on those residuals
by a kernel density estimate of the density fξ.
Finally, the smoothed sieve bootstrap sample
is defined by resampling the AR(p(n)) process from
innovations
:
The purpose of the paper was to carry over a weak dependence property
(here strong mixing) of the initial sequence
to the sieve processes
(a classic and a smoothed version were examined in the paper). The goal
is unrealistic for the classic bootstrap sample because the
distribution of the bootstrapped innovations is discrete. Proving a
mixing property for the smoothed sieve bootstrap sample eludes the
efforts of the authors. In the latter case, it nevertheless appears
that limit theorems can be proved by another method. It consists of
using the following property:
with
and for smooth functions g1,g2
belonging to the classes
(see equation (3.1) in
Bickel and Bühlmann for the definition of the class
and some examples). The new dependence coefficient ν is less than the
strongly mixing coefficient. Bickel and Bühlmann (1999) cannot prove that the sieve sequence
(Xn*) is strongly mixing. A weak dependence
condition is now defined by the ν coefficient. Bickel and Bühlmann
prove that it is satisfied by both this sequence and a smooth version of the
resampled innovations. For instance, Bickel and Bühlmann prove that if
the sequence
satisfies some regularity conditions ensuring that
(recall that νk ≤ αk),
then the sieve bootstrap process
satisfies a ν-mixing condition with a polynomial rate
for relevant classes
and a positive constant L. See Theorem 3.2 in Bickel and
Bühlmann (1999, p. 422) for more
details.
2.5. A Semiparametric Estimation Problem
We follow the presentation in Robinson (1989). He considers an economic variable observable
at time n that is an R × 1 vector of r.v.s
.
We observe Wn at time n = 1
− P,2 − P,…,T where
P is nonnegative and T large. Hypotheses of economic
interest often involve a subset Xn =
B(Wn′,…,Wn−P′)
of the array
(Wn′,…,Wn−P′)′;
for this B is a J × (PR) matrix formed
from the PR-rowed identity matrix
IPR by omitting PR −
J rows (which means that in B, PR −
J elements of
Wn,…,Wn−P
are deleted). Thus, in B, elements of
Wn,Wn−1,…,Wn−P
that are not in Xn are deleted, and
Xn can have elements in common with
Xn+P−1,…,Xn+1,Xn−1,…,Xn−P.
Let Xn =
(Yn′,Zn′)′,
where Yn and
Zn are K × 1 and L
× 1 vectors (K + L = J). The problem
of interest is to test the hypothesis
against the alternative
.
This null hypothesis is written in the form
for
for some M > 0 and some
function H(z,z) defined as
for some convenient function G and where
for any Borel set A of
.
Here f(j)(z) denotes the vector
of j-partial derivatives of f.
An example of this framework is given by
Xn =
(Yn′,Zn′)′,
where Yn =
(tn,sn′)′
and Zn = vn.
The regression model
is of common use in econometrics. Here
sn,tn,vn
are, respectively, scalars, p × 1, and q
× 1; they are observable random sequences whereas the innovation
process (un) is centered and unobservable,
so that
;
we denote
.
In the case of a weakly dependent and stationary innovation process, Robinson
(1989) considers the hypothesis
H0 : β = 0. In this case, the hypothesis can be
written as before, and Robinson calculates β = τ where
K = p + 1, L = 1, M = 0, and
G(x1,x2) =
(t1 −
t2)s1φ(v1)
for some function
(usually φ ≡ 1). Robinson considers the statistics
constructed from the n-sample
(X1,…,Xn). Here,
is a U-statistic and
is the natural estimator of the covariance matrix of
.
One such estimate is
.
Tapered versions might be preferred (see Robinson, 1989, formula (2.21)); here
with di,j =
G(Xi,Xj)k(Zi
− Zj)/h, where
k(z) =
h−L(k(z),h−1k(1)(z)′,…,h−Mk(M)(z)′).
Under β-mixing assumptions, Robinson proves that these estimates are
-consistent and satisfy a CLT. Under a natural β-mixing condition,
Robinson proves in fact that the statistic
has asymptotically a χ2-distribution if
where b > μ/(μ − 2) under the moment assumption
.
The β-mixing assumption allows us to compare the joint
distribution of the initial sequence with respect to a sequence of
r.v.s with independent blocks. This reconstruction is due to
Berbee's coupling lemma, no matter how big the size of the blocks
may be. Yoshihara (1976) derives a covariance
inequality that fits to U-statistics. A way to get rid of
β-mixing conditions is to consider an independent realization
of the trajectory
X1,…,Xn. Now a
simpler estimator of τ is given by
The asymptotic behavior of this expression is easy to derive under
alternative weak dependence conditions by using our results because
is the numerator of a Nadaraya–Watson kernel for the regression
estimation problem
in the special case of the previous example. In fact this trick avoids the
corresponding coupling construction for U-statistics. For
another application of semiparametric problems, see, for example, Chen
and Fan (1999).
2.6. Nonparametric Problems
For a stationary process
with Zt =
(Xt,Yt), an
important quantity is the regression function
.
Various methods to fit such a function have been developed.
Nadaraya–Watson kernel estimates are very popular; see, for instance,
Rosenblatt (1991), Prakasa (1983), and Robinson (1983).
Let K be some kernel function that integrates to 1, Lipschitzian
and with compact support.
Among other problems, one may wish to estimate the volatility of
financial times series, v(x) =
Var(Xt|Xt−1
= x). The question enters the general framework because
v(x) = v2(x) −
v12(x), where
.
Another important problem of econometric interest is to estimate the
marginal density f of a stationary sample. Density kernel
estimators built on K are available. Density and regression
functions of derivatives can also be estimated by using analogous
procedures.
Finally, conditional quantiles are linked to the conditional
distribution
.
More precisely, we denote by q(t|x) =
inf{y; F(y|x) > t}
the generalized (right-continuous, with left limits) inverse of the
monotone function y [map ] F(y|x).
Consistent estimators of the conditional regression
,
where
,
provide information on the previous conditional quantiles.
3. WEAK DEPENDENCE CONDITIONS
Various generalizations of independence have been introduced to
answer the econometric questions discussed in Section 2. The martingale
setting was the first extension of the independence framework (Hall and Heyde, 1980); weakening martingale
conditions yields mixingales. Martingale conditions are written in
terms of conditional expectations, and they seem to be quite
restrictive in econometric practice. NED is a more flexible tool for
modeling fading memory. The ergodic theorem is the first limit theorem
proved for dependent sequences. Another point of view is given by the
mixing properties of stationary sequences in the sense of ergodic
theory: uniform versions of such properties are the forthcoming mixing
properties. Those conditions are also based on independence properties
of the underlying generated σ-algebras. They are also difficult to
check (see Doukhan, 1994).
Our aim is to promote the weak dependence properties, which will be
seen to be much easier to prove.
3.1. Mixing
Let
be a probability space and let
be two sub σ-algebras of
.
Various measures of dependence between
have been introduced; among them we recall
These coefficients are, respectively, the strong mixing coefficient
of Rosenblatt (1956), the absolute regularity
coefficient
of Wolkonski and Rozanov (1959, 1961), the maximal correlation coefficient
of Kolmogorov and Rozanov (1960), and the uniform
mixing coefficient
of Ibragimov (1962).
Let
be a discrete time stationary process. We denote
XA = {Xt;
t ∈ A} the A-marginal of X with
.
Finally, σ(Z) will denote the σ-algebra generated by an r.v.
Z.
For any coefficient previously defined, say, c(.,.), we
shall call the process X a c-mixing process if
limk→∞
cX,k = 0, where
cX,k =
c(σ(X]−∞,0]),σ(X[k,+∞[)).
The following relations hold:
and no reverse implication holds in general. See Doukhan (1994) for more information.
3.2. Mixingales and Near Epoch Dependence
Let
be a real-valued process. We let
.
DEFINITION 3.1 (McLeish, 1975; Andrews, 1988). Let p ≥ 1 and let
be an increasing sequence of σ-algebras. The sequence
is called an
-mixingale
if there exist nonnegative sequences
such that ψ(n) → 0 as n → ∞
and for all integers
,
This property of fading memory is easier to handle than the
martingale condition. A more general concept is the NED on a mixing
process. Its definition can be found in the work by Billingsley (1968), who considered functions of φ-mixing
processes.
DEFINITION 3.2 (Pötscher and Prucha, 1991a, 1991b). Let
p ≥ 1. We consider a c-mixing process
.
For any integers i ≤ j, set
.
The sequence
is called an
-NED
process on the c-mixing process
if there exist nonnegative sequences
such that ψ(n) → 0 as n → ∞
and for all integers
,
This approach is developed in detail in Pötscher and Prucha
(1991). Functions of MA(∞) processes can be
handled using the NED concept. For instance, limit theorems can be deduced
for sums of such functions of MA(∞) processes. These previous
definitions translate the fact that a k-period—ahead in the
first case, both ahead and backward in the second definition—projection
is convergent to the unconditional mean. They are known to be satisfied
by a wide class of models. For example, martingale differences can be
described as
-mixingale
sequences, and linear processes with martingale difference innovations
also.
3.3. Association
The notion of association was introduced independently by Esary,
Proschan, and Walkup (1967) and Fortuin,
Kastelyn, and Ginibre (1971).
The motivations of those authors were radically different because the
first group of authors was working in reliability theory and the others
in mechanical statistics. The condition of the second group of authors
is known as FKG inequality.
DEFINITION 3.3. The sequence
is associated, if for all coordinatewise increasing real-valued functions
h and k,
for all finite disjoint subsets A and B of
.
This extends the positive correlation assumption to model the notion
that two stochastic processes have a tendency to evolve in a similar
way.
This definition is deeper than the simple positive correlatedness.
Besides the evident fact that it does not assume that the variances
exist, one can easily construct orthogonal (hence positively
correlated) sequences that do not have the association property. An
important difference between the preceding conditions is that its
uncorrelatedness implies independence of an associated sequence (Newman, 1984). Let, for instance,
(ξk,ηk) be independent
and i.i.d.
sequences. Then the sequence
defined by Xk =
ξk(ηk −
ηk−1) is neither correlated nor
independent, and hence it is not an associated sequence. Heredity of
association only holds under monotonic transformations. This unpleasant
restriction will disappear under the assumption of weak dependence.
The preceding property of associated sequences was a guideline for
the forthcoming definition of weak dependence. It contains the idea
that weakly correlated associated sequences are also “weakly
dependent.” The very explicit inequality (B.2) proves that this
idea makes sense.
On the opposite side, negatively associated sequences of r.v.s are
defined by a similar relation as the aforementioned covariance
inequality, except for the sign of this inequality. Shao (2000) provides a lucid summary of this type of
association. Then he points out a crucial property of domination by
comparable independent sequences. This property breaks the seemingly
parallel definitions of positively and negatively associated sequences.
We shall develop this idea further.
3.4. Weak Dependence
Here we shall make more explicit the asymptotic independence between
“past” and “future.” Roughly speaking, for
convenient functions h and k, we shall assume that
is small when the distance between the “past” and the
“future” is sufficiently large. Define by
the union of the sets
of numerical bounded measurable functions on some euclidean space
and ∥.∥∞ the corresponding uniform norm. We define
the Lipschitz modulus of a function
by
if x =
(z1,…,zu).
Consider the class
DEFINITION 3.4 (Doukhan and Louhichi,
1999). A sequence
is called
-weak
dependent if there exist a sequence
decreasing to zero at infinity and a function ψ with arguments
such that for any u-tuple
(i1,…,iu) and
any v-tuple
(j1,…,jv) with
i1 ≤ … ≤ iu <
iu + r ≤ j1 ≤
… ≤ jv, one has
if the functions h and k are defined, respectively, on
and on
.
Notice that the sequence θ depends both on the class
and on the function ψ. The function ψ can in fact depend on
all its arguments, contrary to the case of bounded mixing sequences.
This definition is hereditary through images by convenient
functions.
The examples of interest to follow involve the function
ψ1(h,k,u,v) =
uLip(h) + vLip(k),
ψ1′(h,k,u,v)
= vLip(k),
ψ2(h,k,u,v) =
uvLip(h)Lip(k), and
ψ2′(h,k,u,v)
= vLip(h)Lip(k). For example, proving that
an MA(∞) process Xn =
[sum ]k≥
akξn−k
based on an i.i.d. sequence such that
and
[sum ]k|ak|
< ∞ is ψ1′-weakly dependent with
is based on the decomposition Xn =
Xn + Xn
with Xn =
[sum ]k<r
akξn−k.
In this case, assuming for simplicity that v = 1 and
j1 = n, we have
3 Thanks to an anonymous referee, we prove that NED implies
our weak dependence through the following inequalities. For simplicity,
write h =
h(Xi1,…,Xiu),
k =
k(Xj1,…,Xjv),
then the Cauchy–Schwarz inequality gives
and the last expression can be bounded using the
-mixingale
property of
-NED
sequences. Clearly, this implication is not an equivalence between both
notions. It is an open question whether or not these notions are
equivalent.
5. LIMIT THEOREMS
The aim of this section is to present the state of the art of the
limit theorems for stationary sequences.
5.1. The Donsker Line
Consider a stationary sequence
.
We assume that this sequence is integrable and centered at
expectation
Denote by [x] the integer part of a real number
x ([x] ≤ x <
[x] + 1). The Donsker line
(Dn(t))t∈[0,1]
is defined for any sample with positive size n as the
following continuous time process:
We consider the following convergence result in the space
of continuous functions on the unit interval when the sample size n
grows to infinity.
THEOREM 5.1. The following functional convergence holds in the
space
under any of the weak dependence conditions formulated
subsequently:
Here,
(the series is assumed to be convergent).
Recall that here
(Wt)t∈[0,1]
is the standard Brownian motion; that is, W denotes the
centered Gaussian real-valued process with covariance function
To avoid triviality we shall also assume that σ ≠ 0.
The preceding FCLT is known to hold in the cases that follow.
5.1.1. Strong Mixing Case (α-Mixing).
Recall that the quantile function Q of the distribution of
X0 is also the càdlàg (left
continuous with right limits) inverse of the tail function
and that α−1 is the càdlàg
inverse of the monotone function t →
α[t]. The condition
implies the FCLT in Theorem 5.1 and also the convergence of the
series in the definition of σ2 (for more details, see
Rio, 2000). Let δ > 0. Assume that
moment of order (2 + δ) of X0 is finite.
Condition (DMR) is equivalent to
The previous FCLT result was obtained by Davydov (1973) under the slightly stronger assumption
.
In other words, both series converge for the same hyperbolic mixing
decays αn ∼
n−a for a > 1 +
2/δ. Note that no gain seems to be obtained here when one
considers β-mixing sequences.
5.1.2. ρ-Mixing Case.
The condition
with
implies the FCLT, as proven by Shao (1988). It is
well known that the preceding conditions ensure nice behavior of
second-order moments.
5.1.3. Associated Case.
The condition
implies the functional convergence (see Newman and
Wright, 1981). Clearly, this condition is also necessary to
ensure that Theorem 5.1 holds.
Notice that the property “orthogonality implies
independence” makes this condition credible for this very special
case of an associated sequence.
The FCLT is phrased for a strictly stationary negatively associated
sequence in identical terms. In fact, the tightness of
is shown using an exponential inequality that results from a comparison
theorem on moment inequalities proved by Shao (2000).
5.1.4. Nonlinear Functions of Linear Processes.
We consider two-sided linear sequences. If the coefficients in the
linear process
satisfy
,
then
is the decorrelation rate of the sequence. The FCLT holds under the condition
D > 1 (see Giraitis and Surgailis,
1986).
Giraitis and Surgailis (1986) prove the
result for the partial sums of Φ(Xk)
if Φ is some polynomial function with Appell rank m (see
definition that follows). Recall that Appell polynomials are defined
through the relation
Like Hermite polynomials, they satisfy a recursive relation: for all
,
Setting
,
one obtains, for instance,
Let F denote the distribution function of
X0. An analytic function Φ such that
has an uniquely defined Appell expansion:
Now, if the distribution function F is infinitely
differentiable, then, setting f = F′ for the
density function, one obtains
.
A straightforward integration by parts yields
This means that the system of functions
(Ak,Qk)k≥0
is biorthogonal.
It is suitable to define the Appell rank of Φ as the smallest
integer m such that cm ≠ 0.
Appell rank is thus uniquely defined at least for polynomials. The
system of Appell polynomials is not orthogonal (except for the special
case of Hermite polynomials, which are associated with Gaussian
distributions). Hence existence and uniqueness of such expansions
follow from additional conditions such as analyticity. We refer to
Giraitis and Surgailis (1986) for details.
Giraitis and Surgailis (1986) assume the
existence of moments of any order and
.
The functional convergence is ensured by the Chentsov tightness criterion,
given in Appendix B. The reason is that the method of moments is used to
prove the CLT.
Concerning one-sided sequences, Ho and Hsing (1997) obtain an analogous CLT for more general
nonlinear functionals of a one-sided linear sequence. The idea is to
approximate such nonlinear functions of a one-sided linear sequence by
m-dependent moving averages that are easily shown to satisfy an
-CLT.
The main assumptions are
and the following regularity condition. The regularity is twofold. For any
subset J of the set
of nonnegative integers, define
.
A C1-condition on the functions
is required if Card(J) is large enough. Hence, even a very
weak regularity condition on the marginal innovations such as
for a small δ > 0 implies the regularity of the distribution of
YJ, hence the regularity needed on
ΦJ's. Now Burkholder's inequality
for martingales with
still yields the Chentsov tightness criterion.
Such linear sequences are also
-weakly
dependent sequences. Therefore, one may refer to Section 4.3.
5.1.5. Gaussian Processes.
The same finite-dimensional CLT as for linear processes holds for
instantaneous functions of Gaussian stationary sequences when one replaces
m by the Hermite rank of an arbitrary function Φ such that
(see Breuer and Major, 1983).
However, Donsker's Theorem 5.1 requires an additional tightness
condition. Chambers and Slud (1989) introduce
such a condition in terms of the coefficients of Φ's Hermite
expansion. Assume that
,
where
.
This means that an exponential decay of the coefficients is needed to obtain
Donsker's theorem.
Chambers and Slud provide a result for general stationary processes
that are built on a Gaussian chaos. Such functionals may fail to be
instantaneous functions Φ(Yn). They
can be written as general Bernoulli shifts of
:
The authors also consider instantaneous functionals of Gaussian
sequences satisfying CLT but not the Donsker theorem.
Recall, however, that a smooth lower bound assumption on the spectral
density of the process yields ρ-mixing. Hence Theorem 5.1 still
holds under slight additional conditions on the Gaussian process. The
assumption concerning the function Φ is unchanged:
.
5.1.6. (θ,ℒ,ψ)-Weak Dependence.
Assume a
-weak
dependence condition with
,
for the stationary sequence
.
Suppose also that for some
.
Then if the function ψ associated with weak dependence is
ψ1 (respectively, ψ2), Doukhan and
Louhichi (1999) prove the FCLT if
So, without any regularity condition on innovations, Theorem 5.1
holds for a bounded Lipschitz function of a linear process if
when D > 3. The latter doesn't need to be one sided,
whereas Ho and Hsing (1997) need this assumption.
Moreover, the functions Φ available are more general than those
considered by Giraitis and Surgailis (1986).
Finally, a faster hyperbolic decay of coefficients in place of the
boundedness assumption for Φ together with the finiteness of the
fourth-order moment for the innovations yields the FCLT.
Define the class
of real-valued functions
Assuming
-weak
dependence, we can even improve on the preceding condition by the following
uniform bound on the set of integers i ≤ j ≤
k ≤ l and j + r ≤ k
5.1.7. Martingales and Generalizations.
In this section, we consider conditions in terms of conditional
expectations with respect to an adapted filtration. We first recall
that Theorem 5.1 holds for martingales with stationary square
integrable increments such that
(see Billingsley, 1968). More generally, let
be a process adapted to the filtration
is
-measurable
for any
.
The following result is proved by Dedecker and Rio (2000). All ergodicity assumptions are gone. Let
denote the right shift operator (i.e.,
).
Denote by
the tail σ-algebra of T-invariant Borel sets of
.
THEOREM 5.2. Assume that
for any
and
is a convergent series in
.
Denote by
the partial sums. Then the sequence
converges in
to some r.v. η and, conditionally on the tail
σ-algebra
,
the process
converges to the Brownian motion
ηWt.
Remark. This result provides a FCLT with a limit process that is not
Gaussian in general. If the sequence is ergodic, a standard Donsker
theorem holds. Indeed, the ergodicity assumption implies that the r.v.
η is almost surely constant. Hall and Heyde (1980) give this theorem under a more restrictive
assumption: both series
converge in
.
Theorem 5.1 under the condition (DMR) stated in Section
5.1.1 can also be derived as a corollary of Theorem 5.2.
The following corollary can be deduced from Theorem 5.2. Consider a
stationary Markov sequence
with stationary distribution μ and transition operator P. Let
Xn =
g(ξn) be a centered at expectation,
nonlinear functional of
.
Then the FCLT holds under the condition of convergence of the series
.
5.2. The Empirical Cumulative Distribution
Let us consider a stationary sequence
.
We assume without loss of generality that the marginal distribution of
this sequence is uniform on [0,1]. The cumulative
distribution of the empirical process, En,
of the sequence
at time n is defined as
(1/n)En(x), where
We consider the following convergence result in the Skohorod space
when the sample size n converges to infinity:
Here
is the dependent analogue of a Brownian bridge; that is, B
denotes the centered Gaussian process with covariance function given
by
Note that for independent sequences with a marginal cumulative
distribution function F, this just means that
B(x) = B(F(x)) for some
standard Brownian bridge B; this justifies the name
generalized Brownian bridge.
THEOREM 5.3. The following functional convergence holds in the
Skohorod space of real-valued càdlàg functions on the
real line,
,
under the weak dependence conditions detailed in the next sections:
The preceding covariance function can be rewritten as
For the i.i.d. case, it is equivalent to Γ(x,y)
= Γiid(x,y) =
F(x) ∧ F(y) −
F(x)F(y); as the supremum of two
regular functions, this term is intrinsically singular on the diagonal
x = y. This is no longer the case for the other terms
.
If, for instance, the second-order marginals
(X0,Xk) have a
continuous joint density, then
Tk(x,y) is a
C2-function. It is well known that the regularity
properties of a Gaussian process are determined by those of its
covariance function. Hence, the distortion of the Brownian bridge due
to this series is not very important. Either finite-dimensional or
empirical functional convergence (EFCLT) is known to hold in the
following cases.
5.2.1. Strong Mixing Case (α-Mixing).
The condition
implies finite-dimensional convergence. The EFCLT holds if, for some
a > 1,
This result, proved by Rio (2000), improves
on the previous conditions
formerly given by Shao and Yu (1996) and on the
condition a > 3 from Yoshihara (1973).
This condition is close to the previous summability, necessary to ensure
finite-dimensional convergence.
5.2.2. Absolute Regularity Condition
(β-Mixing).
The condition
implies finite-dimensional convergence. Doukhan, Massart, and Rio (1995) obtain the EFCLT Theorem 5.3 when
,
for some a > 2. Here tightness obtains with an additional loss
term of order (log2 n). Finally, Rio (2000) obtains the simple (and optimal) sufficient
condition for EFCLT
In a previous paper Arcones and Yu (1994)
have proved CLTs for empirical processes indexed by so-called V.C.
subgraph classes of functions, not necessarily bounded (for more
details, see van der Vaart and Wellner, 1996,
p. 141). This context contrasts with the common conditions in terms of
bracketing numbers. The EFCLT obtains for uniformly bounded classes in
the pth mean under the β-mixing condition that
5.2.3. ρ-Mixing Case.
The condition
implies finite-dimensional convergence (see Peligrad, 1987). Shao and Yu (1996) obtain the EFCLT under the same condition.
5.2.4. Gaussian Subordinated Case.
Let
be a standard Gaussian stationary process:
.
Consider a function Φ with Hermite rank m such that
.
Csörgő and Mielniczuk (1996)
prove the EFCLT for the subordinated field
Xn =
Φ(Yn) under the natural condition
Hence even if the indicator functions do not satisfy the additional
tightness assumption in Chambers and Slud (1989), the tightness of the empirical process
follows from the diagram formula. In this case the
Kolmogorov–Smirnov statistics are convergent even if the Donsker
theorem does not hold for finite-dimensional distributions of the
empirical cumulative process.
5.2.5. Associated Case.
We assume that the marginal distribution of X0 is
uniform on an interval [0,1]. Then the condition
implies finite-dimensional convergence. Louhichi (2000) obtains the EFCLT Theorem 5.3 under the
condition that, for some a > 4,
Her result improves on the condition of Shao and Yu (1996):
.
5.2.6. One-Sided Linear Processes.
The EFCLT Theorem 5.3 holds if, for some 0 < γ ≤ 1 and
S,C,Δ > 0, with SΔ > 2γ,
we have
If the innovations have moments of any order, the existence of some
Δ > 0 such that the preceding inequality holds implies the
result.
On the other hand, a lower order moment assumption for the innovation
allows higher regularity properties. If, for example, γ = 1 and
S = 2, the inversion formula shows the existence of a
-integrable
density. If now γ = 1 and S = 4, the density must be
-integrable.
For γ = 1, this result recovers the proof of the EFCLT in Doukhan
and Surgailis (1998) when the covariance
series is absolutely convergent. The latter paper considers the case
S = 4; however, Burkholder's inequality for martingales
yields the general result in a straightforward way. Giraitis and
Surgailis (1994) gave a hint of the available
results in a long memory dependence framework.
5.2.7. (θ,ℒ,ψ)-Weak Dependence.
The sequence
is assumed to satisfy a weak dependence condition we now present:
where
(in this case a weak dependence condition holds for a class of functions
working only with the values u = 1 or 2).
PROPOSITION 5.1. Let
be a stationary sequence such that (5.20) holds with
for some ν > 0. Then the sequence of processes
is tight in the Skohorod space
.
In the same way, stationary mixing sequences satisfy the conditions
of Proposition 5.1 if
.
This condition is slightly sharper than Yoshihara's condition,
for some ν > 0. Yet, it is slightly sharper than the corresponding
result in Shao and Yu (see Theorem 2.2 therein), and the result of Rio
(2000) improves on both of them.
THEOREM 5.4. Suppose that
is
-weak
dependent. If either
,
then the empirical functional convergence holds.
Remark. The use of the space
allows one to work with each of the classes of models in the
previous section (association and Gaussian sequences enter the first
case, whereas the second one corresponds to Bernoulli shifts). This
yields new results for Bernoulli shifts and apparently for Markov
sequences. Note that Rio's condition for α-mixing sequences
improves this result. Moreover, Yu (1993)
proves the same result for associated sequences with an exponent loss
term 1.5.
6. FUNCTIONAL ESTIMATION
We consider a stationary process
with Zt =
(Xt,Yt) where
.
The quantity of interest is the regression function
.
Let K be some kernel function integrated to 1, Lipschitzian with
compact support. The kernel estimators are defined by
Here
is a sequence of positive real numbers. We always assume that
hn → 0,
nhn → ∞ as n →
∞.
DEFINITION 6.1. Let ρ = a + b with
.
Define the set of ρ-regular functions
by
Here,
is the set of a-times continuously differentiable functions.
Assuming
,
one can choose a kernel function K of order ρ (not necessarily
nonnegative) such that the bias bh
satisfies
uniformly on any compact subset of
(see Rosenblatt, 1991). If, moreover, ρ is an
integer with b = 1, ρ = a − 1, then with an
appropriately chosen kernel K of order ρ,
bh(x) ∼
(g(ρ)(x)/ρ!)hρ∫
sρK(s) ds, uniformly
on any compact interval.
In view of the asymptotic analysis we assume that the marginal
density f (.) of X0 exists and is
continuous. Moreover, f (x) > 0 for any point
x of interest, and the regression function
exists and is continuous. Finally, for some
exists and is continuous. We set g = fr with obvious
shorthand notation. Moreover, we impose one of the following moment
conditions. Either
or
6.1. Second-Order Properties
We consider first the properties of
.
We also consider the following conditionally centered equivalent of
g2 appearing in the asymptotic variance of the estimator
,
Assume that the densities of the pairs
,
exist and are uniformly bounded: supk>0∥
f(k)∥∞ < ∞.
Moreover, uniformly over all
,
the functions
are continuous. Under these assumptions, the functions
g(k) = f(k)
r(k) are locally bounded.
THEOREM 6.1. Suppose that the stationary sequence
satisfies the conditions (6.23) and (6.24) with p = 2.
Suppose that nδh → ∞ for some
δ ∈]0,1[. Then
under any of the weak dependence condition formulated
subsequently.
To consider asymptotics for the ratio estimator
we use a method, already used by Collomb (1984),
that consists of studying higher order asymptotics. It is the topic of the
next section.
THEOREM 6.2. Suppose that the stationary sequence
satisfies the conditions (6.23) and (6.24) with p = 2.
Consider a positive kernel K. Let
for some ρ ∈]0,2], and
nh1+2ρ
→ 0. Then, for all x belonging to any compact subset of
,
under any of the weak dependence conditions formulated
subsequently.
6.1.1. Strong Mixing Case (α-Mixing).
Theorems 6.1 and 6.2 hold if hn →
0, nhn /log(n) →
∞, and if
for some a > max{6δ,2 + 2δ}. The proof is based on a
Bernstein grouping argument. Besides, Robinson (1983) proves the CLT result in Theorem 5.2 under
condition (5.5) for a > 2S/(S − 2)
without the assumption of positivity of the kernel K.
6.1.2. ρ-Mixing Case.
The estimators in Theorem 6.1 obey the CLT under the mixing assumption
and if the bandwidth condition hn → 0,
nhn /log2(n)
→ ∞ is fulfilled (see Peligrad,
1996).
The latter CLT Theorem 6.2 holds if, for some
and if the bandwidth condition hn → 0,
nhn /log2(n) →
∞, holds. The proof is based on a triangular CLT in Peligrad (1996) combined with moment inequalites B.1 from Shao
(1995).
6.1.3. (θ,ℒ,ψ)-Weak Dependence.
Assuming that the sequence
is
-weak
dependent with
,
for j = 1 or j = 2, Ango Nze et al.
(2002) prove that, uniformly in x
belonging to any compact subset of
,
The exponential moment assumption can be relaxed. Suppose that the
stationary sequence
satisfies conditions (6.22) and (6.24) with p = 2, S
> 2. The former results then hold if the sequence
is
-weak
dependent with
,
for j = 1 or j = 2.
The CLT convergence Theorem 6.1 holds, under the conditions (6.23)
and (6.24) with p = 2 if the stationary sequence
is
-weak
dependent with
and
for j = 1 or j = 2. These results extend the
results of Doukhan and Louhichi (1999), valid
for the case of the density function
,
to the estimate
under weak dependence with either ψ1 or ψ2.
Indeed, the first right-hand-side term is obtained by Bernstein's
blocking technique described in Appendix B. The second right-hand-side term
results from the application of the Lindeberg method (see Rio, 2000).
The CLT convergence Theorem 6.2 relies on the expansion
where
with
Using the Rosenthal inequalities described in Appendix B and the
aforementioned CLT, we obtain the CLT convergence Theorem 6.2 for the
regression function, under conditions (6.23) and (6.24) with p
= 2, if the stationary sequence
is
-weak
dependent with
,
for j = 1 or j = 2.
The results stated in Theorem 6.1 and Theorem 6.2 also hold for
finite-dimensional convergence. The components are asymptotically
jointly independent, much in the same way as for i.i.d. sequences.
Moreover, Rios (1996) proves that the local
linearity test can be handled in the strong mixing case. The function
r is assumed to be ρ continuous. Then the plug-in
estimator
converges to T =
∫r2(x)w(x)
dx if
and the bandwidth condition hn ∈
[n−1/10,n−1/(2ρ−4)].
6.2. Almost Sure Convergence Properties
THEOREM 6.3. Let
be a stationary sequence satisfying the conditions (6.23) and (6.24) with
p = 2. Then under the conditions formulated in the sections
that follow,
Remark. Under condition (ii) of Theorem 6.3, but assuming only the weaker
condition about the bandwidth sequence
we obtain
6.2.1. Strong Mixing Case (α-Mixing).
Liebscher (1996) proves the uniform almost
sure convergence at the optimal rate (εn = 1) if
,
with a > 4 + 3/ρ.
6.2.2. ρ-Mixing Case.
Peligrad (1991) states a uniform almost
sure convergence result with εn =
log(n) if
,
with r > (ρ + 1)/2ρ.
6.2.3. (θ,ℒ,ψ1)-Weak
Dependence Case.
For the sake of simplicity, we only consider the geometrically
dependent case.
THEOREM 6.4. Let
be a stationary sequence satisfying the conditions (6.23) and (6.24) with
p = 2 and either
-weak
dependent with θr ≤
ar for some 0 < a <
1.
