A.1. Proofs for Section 4.

The proof of Theorem 1 consists of three steps. Without loss of generality we deal with the case α = 1; here we will use the subscript 2, for expositional convenience, to denote the nuisance direction. That is, p₂(y_k−1) = p₁(y_k−1) in the case of density function. For component functions, m₂(y_k−1), v₂(y_k−1), and H₂(y_k−1) will be used instead of m₁(y_k−1), v₁(y_k−1), and H₁(y_k−1), respectively. We start by decomposing the estimation errors,

, into the main stochastic term and bias. Use X_n ≃ Y_n to denote X_n = Y_n{1 + o_p(1)} in the following. Let vec(X) denote the vectorization of the elements of the matrix X along with columns.

Proof of Theorem 1.

Step I. Decompositions and Approximations.

Because

is a column vector, the vectorization of equation (3.14) gives

A similar form is obtained for the true function, φ₁(y₁),

by the identity

because

By defining

, the estimation errors are

where

Observing

where

, it follows by adding and subtracting r_k = φ₁(y_k−1) + H₂(y_k−1) that

As a result of the boundedness condition in Assumption A2, the Taylor expansion applied to

at [m(x_k),v(x_k)] yields the first term of τ_n as

where

,

and m*(x_k)[v*(x_k)] is between

. In a similar way, the Taylor expansion of φ₁(y_k−1) at y₁ gives the second term of τ_n as

The term

continues to be simplified by some further approximations. Define the marginal expectation of estimated density functions

as follows:

In the first approximation, we replace the estimated instrument,

, by the ratio of the expectations of the kernel density estimates, p₂(y_k−1)/p(x_k) and deal with the linear terms in the Taylor expansions. That is,

is approximated with an error of

by t_1n + t_2n:

based on the following results:

To show (i), consider the first two elements of the term, for example, which are bounded elementwise by

The last equality is direct from the uniform convergence theorems in Masry (1996) that

and

. The proof for (ii) is shown by applying Lemma A.1, which follows. The negligibility of (iii) follows in a similar way from (ii), considering (A.1). Although the asymptotic properties of s_0n and t_2n are relatively easy to derive, additional approximation is necessary to make t_1n more tractable. Note that the estimation errors of the local linear fits,

, are decomposed into

from the approximation results for the local linear smoother in Jones, Davies, and Park (1994). A similar expression holds for volatility estimates,

, with a stochastic term of (1/n)[sum ]_l [K_h(x_l − x_k)/p(x_l)]v(x_l)(ε_l² − 1). Define

and let J(x_l) denote the marginal expectation of J_k,n with respect to x_k. Then, the stochastic term of t_1n, after rearranging its the double sums, is approximated by

because the approximation error from J(X_l) negligible, that is,

applying the same method as in Lemma A.1. A straightforward calculation gives

where

Observe that (K * K)_i((y_l−1 − y₁)/h) in J(X_l) is actually a convolution kernel and behaves just like a one-dimensional kernel function of y_l−1. This means that the standard method (central limit theorem or law of least numbers) for univariate kernel estimates can be applied to show the asymptotics of

If we define s_1n as the remaining bias term of t_1n, the estimation errors of

consist of two stochastic terms,

, and three bias terms,

, where

Step II. Computation of Variance and Bias.

We start with showing the order of the main stochastic term,

where ξ_k = ξ_1k + ξ_2k,

by calculating its asymptotic variance. Dividing a normalized variance of

into the sums of variances and covariances gives

where the last equality comes from the stationarity assumption.

We claim that

where

Proof of (a). Noting

by the stationarity assumption. Applying the integration with substitution of variable and Taylor expansion, the expectation term is

where κ₃(y₁,z₂) = E [ε_t³|x_t = (y₁,z₂)] and κ₄(y₁,z₂) = E [(ε_t² − 1)²|x_t = (y₁,z₂)]. █

Proof of (b). Because

By setting c(n)h → 0, as n → ∞, we separate the covariance terms into two parts:

To show the negligibility of the first part of the covariances, consider that the dominated convergence theorem used after Taylor expansion and the integration with substitution of variables gives

Therefore, it follows from the assumption on the boundedness condition in Assumption A2 that

where A ≤ B means a_ij ≤ b_ij, for all element of matrices A and B. By the construction of c(n),

Next, we turn to the negligibility of the second part of the covariances:

Let ξ_2kⁱ be the ith element of ξ_2k, for i = 1,…,4. Using Davydov's lemma (in Hall and Heyde, 1980, Theorem A.5), we obtain

for some v > 2. The boundedness of

, for example, is evident from the direct calculation that

Thus, the covariance is bounded by

This implies

if a is such that

for example, c(n)^ah^1−2/v = 1, which implies c(n) → ∞. If we further restrict a such that

then

Thus, c(n)h → 0 as required. Therefore,

as n goes to ∞. █

The proof of (c) is immediate from (a) and (b).

Next, we consider the asymptotic bias. Using the standard result on the kernel weighted sum of the stationary series, we first get

because

For the asymptotic bias of s_1n, we again use the approximation results in Jones et al. (1994). Then, the first component of s_1n, for example, is

and converges to

based on the argument for the convolution kernel given previously. A convolution of symmetric kernels is symmetric, so that ∫(K * K)₀(u)udu = 0 and ∫(K * K)₁(u)u² du = ∫∫wK(w)K(w + u)u² dwdu = 0. This implies that

To calculate s_2n, we use the Taylor series expansion of p₂(y_k−1)/p(X_k):

Thus,

Finally, for the probability limit of

we note that

with

, for i = 0,1,2, and

where q₀ = 1, q₁ = 0, and q₂ = μ_K².

Thus,

. Therefore,

Step III. Asymptotic Normality of

.

Applying the Cramer–Wold device, it is sufficient to show

for all

. We use the small block–large block argument (see Masry and Tjøstheim, 1997). Partition the set {d,d + 1,…,n} into 2k + 1 subsets with large blocks of size r = r_n and small blocks of size s = s_n where

and [x] denotes the integer part of x. Define

Then,

Because of Assumption A6, there exists a sequence a_n → ∞ such that

defining the large block size as

It is easy to show by (A.2) and (A.3) that as n → ∞

We first show that S_n′′ and S_n′′′ are asymptotically negligible. The same argument used in step II yields

which implies

from the condition (A.4). Next, consider

where N_j = j(r + s) + r. Because |N_i − N_j + k₁ − k₂| ≥ r, for i ≠ j, the covariance term is bounded by

The last equality also follows from step II. Hence, (1/n)E {(S_n′′)²} → 0, as n → ∞. Repeating a similar argument for S_n′′′, we get

Now, it remains to show

.

Because η_j is a function of

-measurable, the Volkonskii and Rozanov's lemma (1959) in the appendix of Masry and Tjøstheim (1997) implies that, with

,

where the last two equalities follow from (A.4). Thus, the summands {η_j} in S_n′ are asymptotically independent, because an operation similar to (A.5) yields

Finally, because of the boundedness of density and kernel functions, the Lindeberg–Feller condition for the asymptotic normality of S_n′ holds:

for every δ > 0. This completes the proof of step III.

From

, the Slutzky theorem implies

, where

. In sum,

given by

LEMMA A.1. Assume the conditions in Assumptions A1 and A4–A6. For a bounded function, F(·), it holds that

Proof. The proof of (b) is almost the same as (a). Therefore we only show (a). By adding and subtracting L_l|k(y_l−2|y_k−2), the conditional expectation of L_g(y_l−2 − y_k−2) given y_k−2 in r_1n, we get r_1n = ξ_1n + ξ_2n, where

Rewrite ξ_2n as

where k*(n) is increasing to infinity as n → ∞. Let

which exists as a result of the boundedness of F(x_k). Then, for a large n, the first part of ξ_2n is asymptotically equivalent to (1/n)k*(n)B. The second part of ξ_2n is bounded by

Therefore,

, for example.

It remains to show

. Because E(ξ_1n) = 0 from the law of iteration, we just compute

(1) Consider the case k = i and l ≠ j.

because, by the law of iteration and the definition of L_j|k(y_k−2),

(2) Consider the case l = j and k ≠ i.

We only calculate

because the rest of the triple sum consists of expectations of standard kernel estimates and is O(1/n). Note that

where (L * L)_g(·) = (1/g)∫L(u)L(u + ·/g) is a convolution kernel. Thus, (A.6) is

(3) Consider the case with i = k, j = m:

(4) Consider the case k ≠ i, l ≠ j:

for the same reason as in (1). █

A.2. Proofs for Section 5.

Recall that x_t = (y_t−1,…,y_t−d) = (y_t−α,y_t−α) and z_t = (x_t,y_t). In a similar context, let x = (y₁,..,y_d) = (y_α,y_α) and z = (x,y₀). For the score function s*(z,θ,γ_α) = s*(z,θ,γ_α(y_α)), we define its first derivative with respect to the parameter θ by

and use

to denote E [s*(z_t,θ,γ_α)] and E [∇_θ s*(z_t,θ,γ_α)] , respectively. Also, the score function s*(z,θ,·) is said to be Frechet differentiable (with respect to the sup norm ∥·∥_∞) if there is S*(z,θ,γ_α) such that for all γ_α with ∥γ_α − γ_α⁰∥_∞ small enough,

for some bounded function b(·). The term S*(z,θ,γ_α⁰) is called the functional derivative of s*(z,θ,γ_α) with respect to γ_α. In a similar way, we define ∇_γ S*(z,θ,γ_α) to be the functional derivative of S*(z,θ,γ_α) with respect to γ_α.

Assumption B. Suppose that (i)

is nonsingular; (ii) S*(z,θ,γ_α(y_α)) and ∇_γ S*(z,θ,γ_α(y_α)) exist and have square integrable envelopes S*(·) and ∇_γ S*(·), satisfying

and (iii) both s*(z,θ,γ_α) and S*(z,θ,γ_α) are continuously differentiable in θ, with derivatives bounded by square integrable envelopes.

Note that the first condition is related to the identification condition of component functions, whereas the second concerns Frechet differentiability (up to the second order) of the score function and uniform boundedness of the functional derivatives. For the main results in Section 5, we need the following conditions. Some of the assumptions are stronger than their counterparts in Assumption A in Section 4. Let h₀ and h denote the bandwidth parameter used for the preliminary instrumental variable and the two-step estimates, respectively, and g denote the bandwidth parameter for the kernel density.

Assumption C.

2. The joint density function, p(·), is bounded away from zero and q-times continuously differentiable on the compact supports

, with Lipschitz continuous remainders, that is, there exists C < ∞ such that for all

, for all vectors μ = (μ₁,…,μ_d) with

.

3. The component functions, m_α(·) and v_α(·), for α = 1,…,d, are q-times continuously differentiable on

with Lipschitz continuous qth derivative.

8. (i)

and for some integer ω > d/2,

Some facts about empirical processes will be useful in the discussion that follows. Define the L²-Sobolev norm (of order q) on the class of real-valued function with domain

:

where, for

and a k-vector μ = (μ₁,…,μ_k) of nonnegative integers,

and q ≥ 1 is some positive integer. Let

be an open set in

with minimally smooth boundary as defined by, for example, Stein (1970), and

. Define

as a class of smooth functions on

whose L²-Sobolev norm is bounded by some constant

. In a similar way,

.

Define (i) an empirical process, v_1n(·), indexed by

:

with pseudometric ρ₁(·,·) on

:

where f₁(w;τ) = h^−1/2K((w_α − y_α)/h)S*(w,γ_α⁰)τ₁(w_α); and (ii) an empirical process, v_2n(·,·), indexed by

:

with pseudometric ρ₂(·,·) on

:

where f₂(w; y_α,τ₂) = h₀^−1/2K [(w_α − y_α)/h₀][p_α(w_α)/p(w)]G_m′(m(w))τ₂(w).

We say that the processes {ν_1n(·)} and {ν_2n(·,·)} are stochastically equicontinuous at τ₁⁰ and (y_α⁰,τ₂⁰), respectively (with respect to the pseudometric ρ₁(·,·) and ρ₂(·,·), respectively), if

and

respectively, where P* denotes the outer measure of the corresponding probability measure.

Let

be the class of functions such as f₁(·) defined previously. Note that (A.10) follows, if Pollard's entropy condition is satisfied by

with some square integrable envelope F₁; see Pollard (1990) for more details. Because f₁(w;τ₁) = c₁(w)τ₁(w_α) is the product of smooth functions τ₁ from an infinite-dimensional class (with uniformly bounded partial derivatives up to order q) and a single unbounded function c(w) = [h^−1/2K((w_α − y_α)/h)S*(w,γ_α⁰)] , the entropy condition is verified by Theorem 2 in Andrews (1994) on a class of functions of type III. Square integrability of the envelope F₁ comes from Assumption B(ii). In a similar way, we can show (A.11) by applying the “mix and match” argument of Theorem 3 in Andrews (1994) to f₂(w; y_α,τ₂) = c₂(w)h^−1/2K((w_α − y_α)/h₀)τ₂(w), where K(·) is Lipschitz continuous in y_α, that is, a function of type II.

Proof of Theorem 4. We only give a sketch, because the whole proof is lengthy and relies on arguments similar to Andrews (1994) or Gozalo and Linton (2000) for the i.i.d. case. Expanding the first-order condition in (5.16) and solving for

yields

where θ is the mean value between

. By the uniform law of large numbers in Gozalo and Linton (2000), we have

, which, together with (i) uniform convergence of

by Lemma A.3 and (ii) uniform continuity of the localized likelihood function, Q_n(θ,γ_α) over Θ × Γ_α, yields

and thus consistency of

. Based on the ergodic theorem on the stationary time series and a similar argument to Theorem 1 in Andrews (1994), consistency of

and uniform convergence of

imply

For the numerator, we first linearize the score function. Under Assumption B(ii), s*(z,θ,γ_α) is Frechet differentiable and (A.7) holds, which, because of

(by Lemma A.3 and Assumption C.8(i)), yields a proper linearization of the score term:

where S*(z_t,γ_α⁰(y_t−α)) = S*(z_t,θ₀,γ_α⁰(y_t−α)). Or equivalently, by letting

and u_t = S*(x_t,γ_α⁰(y_t−α)) − E [S*(x_t,γ_α⁰(y_t−α))|x_t = y] , we have

Note that the asymptotic expansion of the infeasible estimator is equivalent to the first term of the linearized score function premultiplied by the inverse Hessian matrix in (A.12). Because of the asymptotic boundedness of (A.12), it suffices to show the negligibility of the second and third terms.

To calculate the asymptotic order of T_2n, we make use of the preceding stochastic equicontinuity results. For a real-valued function δ(·) on

, we define an empirical process

where f (x_t; y_α,δ) = K((y_t−α − y_α)/h)h^ωS*(x_t,γ_α⁰(y_t−α))δ(y_t−α), for some integer ω > d/2. Let

. From the uniform convergence rate in Lemma A.3 and the bandwidth condition C.8(ii), it follows that

Because

is bounded uniformly over

, with probability approaching one, it holds that

. Also, because, for some positive constant C < ∞,

we have

. Hence, following Andrews (1994, p. 2257), the stochastic equicontinuity condition of v_n(y_α,·) at δ⁰ = 0 implies that

; that is, T_2n is approximated (with an o_p(1) error) by

We proceed to show negligibility of T_n2*. From the integrability condition on S*(z,γ_α⁰(y_α)), it follows, by change of variables and the dominated convergence theorem, that ∫K_h(y_α − y_α⁰)S*(z,γ_α⁰(y_α)) dF₀(z) = ∫S*[(y,y_α⁰,y_α),γ_α⁰(y_α)] × p(y,y_α⁰,y_α) d(y,y_α) < ∞, which, together with

-consistency of

, means that

. Because

this yields

From Lemma A.3,

where

. Under the condition C.8(i),

, integrability of the bias function b_β(y_β) and S*(z,θ₀,γ_α⁰(y_α)) imply

where

Let

be the ith elements of

, respectively, with S*^ij(·) being the (i,j) element of S*(·). By the dominated convergence theorem and the integrability condition, we have

where

and ∇G^j(·) = ∇G_m(·), for j = 1; ∇G_v(·), for j = 2. Because p₂(·)/p(·) and

are bounded under the condition of compact support, applying the law of large numbers for i.i.d. errors

leads to

and consequently

. Likewise,

where

and, for the same reason as before, we get

, because E(m_α(y_t−α)) = E(v_α(y_t−α)) = 0.

We finally show negligibility of the last term:

Substituting the error decomposition for

and interchanging the summations gives

where the o_p(1) errors for the remaining bias terms hold under the assumption that

. For

we can easily check that E(π_1n^i,β(z_t,z_s)|z_t) = E(π_1n^i,β(z_t,z_s)|z_s) = 0, for t ≠ s, implying that [sum ][sum ]_t≠s π_n^i,β(z_t,z_s) is a degenerate second-order U-statistic. The same conclusion also holds for the second term. Hence, the two double sums are mean zero and have variance of the same order as

which is of order n⁻¹h⁻¹. Therefore, T_3n = o_p(1). █

LEMMA A.2. (Masry, 1996). Suppose that Assumption C holds. Then, for any vector

with |μ| = Σ_j μ_j ≤ ω,

LEMMA A.3. Suppose that Assumption C holds. Then, for any vector

with |μ| = Σ_j μ_j ≤ ω,

Proof. We first show (b). For notational simplicity, the bandwidth parameter h (only in this proof) abbreviates h₀. From the decomposition results for the instrumental variable estimates,

By the Cauchy–Schwarz inequality and Lemma A.2 applied with Taylor expansion, it holds that

where the boundedness condition of C.2 is used for the last line. Hence, the standard argument of Masry (1996) implies that

, where q_i = ∫K(u₁)u₁ⁱ du₁. From q₀ = 1, q₁ = 0, and q₂ = μ_K², we get the following uniform convergence result for the denominator term; that is,

, uniformly in

. For the numerator, we show the uniform convergence rate of the first element of τ_n because the other terms can be treated in the same way. Let τ_n¹ denote the first element of τ_n, that is,

or alternatively,

where

Because p_α(·)/p(·) is bounded away from zero and G_m has a bounded second-order derivative, the functional r(x_t;g) is Frechet differentiable in g, with respect to the sup norm ∥·∥_∞, with the (bounded) functional derivative R(x_t;g) = [∂r(x_t;g)/∂g]↓_g=g(xt). This implies that for all g with ∥g − g⁰∥_∞ small enough, there exists some bounded function b(·) such that

By Lemma A.2,

, and consequently, we can properly linearize τ_n¹ as

where the O_p(ρ_n²) error term is uniformly in x_α. After plugging G_m(m(x_t)) = c_m + Σ_1≤β≤d m_β(y_t−β) into r(x_t;g⁰), a straightforward calculation shows that

where ς_t = [p₂(y_t−α)/p(x_t)]M_α(y_t−α) and M_α(y_t−α) = Σ_{1≤β≤d,(≠α)} m_β(y_t−α). Note that as a result of the identification condition E [ς_t|y_t−α] = 0 and consequently the first term is a standard stochastic term appearing in kernel estimates. For a further asymptotic expansion of the second term of τ_n¹, we use the stochastic equicontinuity argument to the empirical process {v_n(·,·)}, indexed by

, with

, such that

where f (x_t; y_α,δ) = K[(y_t−α − y_α)/h] h^ω[p_α(y_t−α)/p(x_t)]G_m′(m(x_t))δ(y_t−α), for some positive integer ω > d/2. Let

. From the uniform convergence rate in Lemma A.2 and the bandwidth condition in C.8(iii), it follows that

, leading to (i)

and (ii)

, where δ⁰ = 0. These conditions and stochastic equicontinuity of v_n(·,·) at (y_α,δ⁰) yield

. Thus, the second term of τ_n¹ is approximated with an

error (uniform in y_α) by

which, by substituting

, is given by

where (K * K)(·) is actually a convolution kernel as defined before. Hence, by letting b_α(y_α) summarize two bias terms appearing in (A.13) and (A.14), Lemma A.3(b) is shown. The uniform convergence results in part (a) then follow by the standard arguments of Masry (1996), because two stochastic terms in the asymptotic expansion of

consist only of univariate kernels. █