2.1 Symplectic group

First we introduce some notation for symplectic groups. Let $(W, \langle -,- \rangle _W)$ be a symplectic vector space of dimension $2n$ over $F$, with associated symplectic group

\[ {\rm Sp}(W) = \left\{ { g \in {\rm GL}(W) | \langle gw, gw' \rangle_W = \langle w, w' \rangle_W \ \forall \, w, w' \in W } \right\}. \]

Choose a symplectic basis $\left\{ {y_1, \ldots , y_n, y^*_1, \ldots , y^*_n} \right\}$ of $W$, and define

\[ Y_k = {{\rm span}}_F(y_1, \ldots, y_k),\quad Y^*_k = {{\rm span}}_F(y^*_1,\ldots, y^*_k) \]

for $k=1, \ldots , n$, so that we have a standard complete polarization $W = Y_n \oplus Y^*_n$. We also let

\[ W_{n-k}= {{\rm span}}_F(y_{k+1}, \ldots,y_n, y^*_{k+1}, \ldots y^*_n) \]

so that

\[ W = Y_k \oplus W_{n-k} \oplus Y^*_k. \]

If $n=0$, then $W=\{0\}$, ${\rm Sp}(W)=\{1\}$, and the basis is the empty set.

We now describe the parabolic subgroups of ${\rm Sp}(W)$ up to conjugacy. Let $\mathbf{k}=(k_1, \ldots , k_m)$ be a sequence of positive integers such that $k_1+\cdots +k_m \leq n$, and put $k_0 = 0$ and $n_0 = n-(k_1+\cdots +k_m)$. Consider a flag of isotropic subspaces

\[ Y_{k_1} \subset Y_{k_1+k_2} \subset \cdots \subset Y_{k_1+\cdots +k_m} \]

in $Y_n$. The stabilizer of such a flag is a parabolic subgroup $\overline {P_\mathbf{k}}$ whose Levi subgroup $\overline {M_\mathbf{k}}$ is given by

\[ \overline{M_\textbf{k}} \cong {\rm GL}_{k_1} \times \cdots \times {\rm GL}_{k_m} \times {\rm Sp}(W_{n_0}), \]

where ${\rm GL}_{k_i}$ is identified with the general linear group of a $k_i$-dimensional space

\[ {{\rm span}}_F(y_{k_0+\cdots+k_{i-1}+1}, \ldots,y_{k_0+\cdots+k_{i-1}+k_i}). \]

The reason we use the overlines for $\overline {M_\mathbf{k}}$ and $\overline {P_\mathbf{k}}$ will become clear in the next subsection. We shall write $N_\mathbf{k}$ for the unipotent radical of $\overline {P_\mathbf{k}}$. Parabolic subgroups of this form are standard with respect to the splitting $\textbf {spl}_{{\rm Sp}(W)}$ defined in § 7.1. Any parabolic subgroup of ${\rm Sp}(W)$ is conjugate to a parabolic subgroup of this form. If $m=1$ and $\mathbf{k}=(k)$, we shall write $\overline {P_k}$, $\overline {M_k}$, and $N_k$ instead of $\overline {P_\mathbf{k}}$, $\overline {M_\mathbf{k}}$, and $N_\mathbf{k}$, respectively, for simplicity.

2.2 Metaplectic group

Next we come to metaplectic groups. If $n=0$, we put ${\rm Mp}(W)=\{\pm 1\}$. If $n\geq 1$, then the symplectic group ${\rm Sp}(W)$ has a unique nonlinear two-fold central extension ${\rm Mp}(W)$, which is called the metaplectic group:

(2.1)\begin{equation} 1 \longrightarrow \{\pm1\} \longrightarrow {\rm Mp}(W) \longrightarrow {\rm Sp}(W) \longrightarrow 1. \end{equation}

As a set, we may write

\[ {\rm Mp}(W) ={\rm Sp}(W) \times\{\pm1\} \]

with group law given by

\[ (g,\epsilon) \cdot (g',\epsilon') = (gg', \epsilon\epsilon' c(g,g')), \]

where $c$ is Ranga Rao's normalized cocycle, which is a 2-cocycle on ${\rm Sp}(W)$ valued in $\{\pm 1\}$. See [Reference Ranga RaoRan93, § 5] or [Reference SzpruchSzp07, § 2] for details. For any subset $A \subset {\rm Sp}(W)$, we write $\widetilde {A}$ for its preimage under the covering map ${\rm Mp}(W) \to {\rm Sp}(W)$. Also, for any subset $B \subset {\rm Mp}(W)$, we write $\overline {B}$ for its image under the covering map.

By the parabolic subgroups of ${\rm Mp}(W)$ and their Levi subgroups we mean the preimages of the parabolic subgroups of ${\rm Sp}(W)$ and their Levi subgroups, respectively. Not only the metaplectic group ${\rm Mp}(W)$ but also its parabolic subgroups and Levi subgroups are in general nonlinear.

Let us describe the parabolic subgroup $P_\mathbf{k} = \widetilde {\overline {P_\mathbf{k}}}$ of ${\rm Mp}(W)$, which we shall call a standard parabolic subgroup (with respect to the splitting $\textbf {spl}_{{\rm Sp}(W)}$). The covering (2.1) splits over the unipotent radical $N_\mathbf{k}$ of $\overline {P_\mathbf{k}}$ by $n \mapsto (n, 1)$, so we may canonically regard $N_\mathbf{k}$ as a subgroup of ${\rm Mp}(W)$, and we have a Levi decomposition

\[ P_\textbf{k}= M_\textbf{k} \ltimes N_\textbf{k} \]

where $M_\mathbf{k} = \widetilde {\overline {M_\mathbf{k}}}$ is a Levi subgroup. The covering $M_\mathbf{k}$ over $\overline {M_\mathbf{k}} \cong {\rm GL}_{k_1} \times {\rm GL}_{k_2} \times \cdots \times {\rm GL}_{k_m} \times {\rm Sp}(W_{n_0})$ is given by

\[ M_\textbf{k} \cong \widetilde{{\rm GL}}_{k_1} \times_{\mu_2} \cdots \times_{\mu_2} \widetilde{{\rm GL}}_{k_m} \times_{\mu_2} {\rm Mp}(W_{n_0}). \]

Here, the restriction of the covering to ${\rm Sp}(W_{n_0})$ is nothing but the metaplectic cover ${\rm Mp}(W_{n_0})$ of ${\rm Sp}(W_{n_0})$, and the covering over ${\rm GL}_{k_i}$ is

\[ \widetilde{{\rm GL}}_{k_i}={\rm GL}_{k_i} \times \left\{ {\pm1} \right\} \]

with group law

\[ (g,\epsilon) \cdot (g',\epsilon') = (gg', \epsilon \epsilon' (\det g, \det g')_F). \]

Let $k$ be a positive integer. The (genuine) representation theory of $\widetilde {{\rm GL}}_k$ can be easily related to the representation theory of ${\rm GL}_k$. Indeed, to any irreducible representation $\tau$ of ${\rm GL}_k$ we can attach an irreducible genuine representation $\widetilde {\tau }$ of $\widetilde {{\rm GL}}_k$ as in [Reference Gan and SavinGS12, § 2.4], and this attachment $\tau \mapsto \widetilde {\tau }$ gives a bijection between ${\rm Irr}({\rm GL}_k)$ and ${\rm Irr}(\widetilde {{\rm GL}}_k)$, where ${\rm Irr}(\widetilde {{\rm GL}}_k)$ is the set of equivalence classes of irreducible genuine representations of $\widetilde {{\rm GL}}_k$. We stress that this bijection depends on the choice of the additive character $\psi$ because $\widetilde {\tau }$ is the twist $\tau \otimes \chi _\psi$ by a genuine character $\chi _\psi$, which is defined using $\psi$, as in [Reference Gan and SavinGS12, § 2.4].

2.3 Orthogonal group

Now we come to the orthogonal groups. Let $V$ be a $(2n+1)$-dimensional vector space over $F$ equipped with a non-degenerate quadratic form $q=q_V$ of discriminant $1$. Then we define a symmetric bilinear form $b_q$ associated to $q$ by

\[ b_q(v,v') = q(v+v') - q(v) -q(v'). \]

If $n \geq 1$, then up to isomorphism there are precisely two such quadratic spaces $V$. One of them, denoted by $V^+$, has maximal isotropic subspaces of dimension $n$, whereas the other, denoted by $V^-$, has maximal isotropic subspaces of dimension $n-1$. As such, we call the former the split quadratic space and the latter the non-split quadratic space. We shall write

\[ \epsilon(V) =\begin{cases} +1, & V=V^+, \\ -1, & V=V^-. \end{cases} \]

If $n=0$, we have only one such $V$ up to isomorphism, and we put $V^+= V$ and $\epsilon (V)=+1$.

Let

\[ {\rm O}(V)= \left\{ { h \in {\rm GL}(V) | q(hv) = q(v)\,\ \forall \, v \in V} \right\} \]

be the associated orthogonal group. Then observe that ${\rm O}(V) = {\rm SO}(V) \times \{\pm 1\}$, where

\[ {\rm SO}(V)={\rm O}(V) \cap {\rm SL}(V) \]

is the special orthogonal group. The group ${\rm SO}(V)$ is split (respectively non-quasi-split) if $V$ is the split (respectively non-split) quadratic space. If $n \geq 1$, then up to isomorphism there are precisely two pure inner twists of ${\rm SO}(V^+)$, namely ${\rm SO}(V^+)$ and ${\rm SO}(V^-)$. Note that the Kottwitz sign [Reference KottwitzKot83] of ${\rm SO}(V)$ is equal to $\epsilon (V)$.

Let $r$ be the dimension of a maximal isotropic subspace of $V$, so that $r=n-(1-\epsilon (V))/2$. Choose a basis $\left\{ {x_1, \ldots , x_n, x_0, x^*_1, \ldots , x^*_n} \right\}$ of $V$ such that

\begin{align*} &b_q(x_i, x_j) = b_q(x^*_i, x^*_j) = 0,\quad b_q(x_i, x^*_j) = \delta_{i,j}, \\ &b_q(x_0, x_i) = b_q(x_0,x^*_i) =0,\quad q(x_0) = 1 \end{align*}

for $1\leq i,j \leq r$ and, if $r=n-1$,

\[ b_q(x_n,x_i)=b_q(x_n,x^*_i)=b_q(x^*_n,x_i) =b_q(x^*_n, x^*_i)=0 \]

for any $1\leq i \leq r$. For each $1\leq k\leq r$, put

\begin{gather*} X_k = {{\rm span}}_F(x_1, \ldots, x_k),\quad X^*_k = {{\rm span}}_F(x^*_1,\ldots, x^*_k),\\ V_{n-k} = {{\rm span}}_F(x_{k+1}, \ldots,x_n, x_0, x^*_{k+1}, \ldots x^*_n), \end{gather*}

so that

\[ V =X_k \oplus V_{n-k} \oplus X^*_k. \]

We now describe the parabolic subgroups of ${\rm SO}(V)$ up to conjugacy. Let $\mathbf{k}=(k_1,\ldots , k_m)$ be a sequence of positive integers such that $k_1+\cdots +k_m \leq r$. Put $k_0 = 0$ and $n_0 = n - (k_1+ \cdots + k_m)$. Consider a flag of isotropic subspaces

\[ X_{k_1} \subset X_{k_1+k_2} \subset \cdots \subset X_{k_1+\cdots +k_m} \]

in $X_r$. The stabilizer of such a flag is a parabolic subgroup $Q_\mathbf{k}$ whose Levi subgroup $L_\mathbf{k}$ is given by

\[ L_\textbf{k} \cong {\rm GL}_{k_1} \times \cdots \times {\rm GL}_{k_m} \times {\rm SO}(V_{n_0}), \]

where ${\rm GL}_{k_i}$ is identified with the general linear group of a $k_i$-dimensional space

\[ {{\rm span}}_F(x_{k_0+\cdots+k_{i-1}+1}, \ldots,x_{k_0+\cdots+k_{i-1}+k_i}). \]

We shall write $U_\mathbf{k}$ for the unipotent radical of $Q_\mathbf{k}$. Parabolic subgroups of this form are standard with respect to the splitting $\textbf {spl}_{{\rm SO}(V^+)}$ defined in § 7.1 if $V=V^+$. Any parabolic subgroup of ${\rm SO}(V)$ is conjugate to a parabolic subgroup of this form.

3.1 Symplectic representations of $\textit {WD}_F$ and their component groups

We say that a homomorphism $\phi : \textit {WD}_F \to {\rm GL}_d(\mathbb {C})$ is a representation of $\textit {WD}_F$ if:

1. – $\phi ({\rm Frob})$ is semi-simple, where ${\rm Frob} \in W_F$ is a geometric Frobenius;

2. – the restriction of $\phi$ to ${\rm SL}_2(\mathbb {C})$ is algebraic;

3. – the restriction of $\phi$ to $W_F$ is smooth.

We say that $\phi$ is tempered if the image of $W_F$ is bounded, and we say that $\phi$ is symplectic if there exists a non-degenerate anti-symmetric bilinear form $B : \mathbb {C}^d \times \mathbb {C}^d \to \mathbb {C}$ such that $B(\phi (w)x, \phi (w)y) = B(x, y)$ for any $x, y \in \mathbb {C}^d$ and $w \in \textit {WD}_F$; in this case, $\phi$ is self-dual.

Let $\phi : \textit {WD}_F \to {\rm GL}_d(\mathbb {C})$ be a tempered symplectic representation. By changing bases if necessary, we may assume that $\phi : \textit {WD}_F \to {\rm Sp}_d(\mathbb {C})$. Then, by [Reference Gan, Gross and PrasadGGP12, § 4], we can write

\[ \phi =\bigoplus_{i\in I_\phi} \ell_i \phi_i \oplus (\varphi \oplus \varphi^\vee), \]

where the $\ell _i$ are positive integers, $I_\phi$ is an indexing set for mutually non-equivalent irreducible symplectic representations $\phi _i$ of $\textit {WD}_F$, and $\varphi$ is a representation of $\textit {WD}_F$ such that all irreducible summands are non-symplectic. Let $S_\phi = {{\rm Cent}}({{\rm Im}} \phi , {\rm Sp}_d(\mathbb {C}))$ be the centralizer of the image ${{\rm Im}}(\phi )$ in ${\rm Sp}_d(\mathbb {C})$. Then, by [Reference Gan, Gross and PrasadGGP12, § 4], its component group $\pi _0(S_\phi )$ is canonically identified with a free $\mathbb {Z}/2\mathbb {Z}$-module of rank $\#I_\phi$:

\[ \pi_0(S_\phi) \cong \bigoplus_{i \in I_\phi} \, (\mathbb{Z}/2\mathbb{Z}) a_i, \]

where $\{a_i\}$ is a formal basis associated to $\{\phi _i\}$. In the rest of this paper, we identify $\pi _0(S_\phi )$ with $\bigoplus _i(\mathbb {Z}/2\mathbb {Z})a_i$. We shall write $z_\phi$ for the image of $-1\in S_\phi$ in $\pi _0(S_\phi )$.

3.2 Tempered $L$-parameters for ${\rm Mp}(W)$ and ${\rm SO}(V)$

Let ${\Phi }_{\rm temp}({\rm GL}_k)$ be the set of equivalence classes of tempered $L$-parameters for ${\rm GL}_k$. Recall that it can be identified with the set of equivalence classes of tempered representations $\phi : \textit {WD}_F \to {\rm GL}_k(\mathbb {C})$ of dimension $k$. Now let ${\Phi }_{\rm temp}({\rm Mp}_{2n})$ and ${\Phi }_{\rm temp}({\rm SO}_{2n+1})$ be the sets of equivalence classes of tempered $L$-parameters for ${\rm Mp}(W)$ and ${\rm SO}(V)$, respectively. Then, by [Reference Gan, Gross and PrasadGGP12, §§ 11 and 8], we can identify ${\Phi }_{\rm temp}({\rm Mp}_{2n})$ and ${\Phi }_{\rm temp}({\rm SO}_{2n+1})$ with the set of equivalence classes of tempered symplectic representations $\phi : \textit {WD}_F \to {\rm Sp}_{2n}(\mathbb {C})$ of dimension $2n$.

Let $\mathbf{k} = (k_1,\ldots ,k_m)$ and let $n_0$ be as in the previous section. For $(G,P,M) =({\rm Mp}(W), P_\mathbf{k}, M_\mathbf{k})$ or $({\rm SO}(V), Q_\mathbf{k}, L_\mathbf{k})$, put

\[ \hat{G} = {\rm Sp}_{2n}(\mathbb{C}),\quad \hat{M} = {\rm GL}_{k_1}(\mathbb{C}) \times \cdots \times {\rm GL}_{k_m}(\mathbb{C}) \times {\rm Sp}_{2n_0}(\mathbb{C}), \]

with a standard embedding $\hat {M} \hookrightarrow \hat {G}$ as a Levi subgroup of a standard parabolic subgroup $\hat {P}$ of $\hat {G}$. Let $\phi$ be a tempered $L$-parameter for $G$ with the image ${{\rm Im}}(\phi )$ in $\hat {M}$. This is of the form

(3.1)\begin{equation} \phi = \phi_1 \oplus \cdots \oplus \phi_m \oplus \phi_0 \oplus \phi_m^\vee \oplus \cdots \oplus \phi_1^\vee, \end{equation}

where $\phi _i \in {\Phi }_{\rm temp}({\rm GL}_{k_i})$ for $i=1,\ldots ,m$ and $\phi _0 \in {\Phi }_{\rm temp}({\rm Mp}_{2n_0})={\Phi }_{\rm temp}({\rm SO}_{2n_0+1})$. Let $A_{\hat {M}}$ be the maximal central torus of $\hat {M}$. Put

\begin{align*} \mathfrak{N}_\phi(M, G) &= {{\rm Norm}}(A_{\hat{M}}, S_\phi) / {{\rm Cent}}(A_{\hat{M}}, S_\phi^\circ),\\ {\rm W}_\phi(M, G) &= {{\rm Norm}}(A_{\hat{M}}, S_\phi) / {{\rm Cent}}(A_{\hat{M}}, S_\phi),\\ S_\phi^\natural(M, G) &= {{\rm Norm}}(A_{\hat{M}}, S_\phi) / {{\rm Norm}}(A_{\hat{M}}, S_\phi^\circ). \end{align*}

We have a natural surjection

(3.2)\begin{equation} \mathfrak{N}_\phi(M,G) \twoheadrightarrow S_\phi^\natural(M,G), \end{equation}

natural inclusions

\[ {\rm W}_\phi(M,G) \subset {\rm W}(\hat{M}, \hat{G}),\quad S_\phi^\natural(M, G) \subset \pi_0(S_\phi), \]

and a natural short exact sequence

\[ 1 \longrightarrow \pi_0(S_{\phi_0}) \longrightarrow \mathfrak{N}_\phi(M, G) \longrightarrow {\rm W}_\phi(M, G) \longrightarrow 1. \]

By applying [Reference ArthurArt13, p. 104] or [Reference Kaletha, Mínguez, Shin and WhiteKMSW14, p. 103, after (2.4.1)] to ${\rm SO}(V)$, the injection $\pi _0(S_{\phi _0}) \to \mathfrak {N}_\phi (M, G)$ admits a canonical splitting

\[ \mathfrak{N}_\phi(M, G) = \pi_0(S_{\phi_0}) \times {\rm W}_\phi(M, G). \]

6.1 Weil representation

The group ${\rm Mp}(W) \times {\rm O}(V)$ has a natural representation $\omega _{V, W, \psi }$ depending on $\psi$, given as follows. The tensor product $\mathbb {W}=V \otimes _F W$ has a natural symplectic form $\langle -,- \rangle$ defined by

\[ \langle v\otimes w, v'\otimes w' \rangle = b_q(v, v') \cdot \langle w, w' \rangle_W. \]

Then there is a natural map

(6.1)\begin{equation} {\rm Sp}(W) \times {\rm O}(V) \longrightarrow {\rm Sp}(\mathbb{W}). \end{equation}

One has the metaplectic $\mathbb {C}^1$-cover $\mathcal {M}p(\mathbb {W})$ of ${\rm Sp}(\mathbb {W})$, and the additive character $\psi$ determines the Weil representation $\omega _\psi$ of $\mathcal {M}p(\mathbb {W})$. Kudla [Reference KudlaKud94] gives a splitting of the metaplectic cover over ${\rm Mp}(W) \times {\rm O}(V)$, and hence there exists a commutative diagram

where the right vertical map is given by the metaplectic $\mathbb {C}^1$-covering map, the left vertical map is given by the two-fold cover (2.1), and the lower horizontal map is (6.1). Thus, we have a Weil representation $\omega _{V, W, \psi }$ of ${\rm Mp}(W) \times {\rm O}(V)$. Later, in § 8.6, we will give some realizations of the Weil representation $\omega _{V, W, \psi }$ to show the main theorem. Here, a splitting over ${\rm Mp}(W) \times {\rm O}(V)$ is not unique, and we choose one following [Reference KudlaKud94].

6.2 Local theta correspondence

In this subsection, we summarize the Gan–Savin result [Reference Gan and SavinGS12]. First, note that the theorems in [Reference Gan and SavinGS12] had been verified only for the case of odd residue characteristic, as the Howe duality for even residue characteristic was conjecture at the time. However, the Howe duality for even residue characteristic was then verified by Gan and Takeda [Reference Gan and TakedaGT16], so now we have the results of [Reference Gan and SavinGS12] for arbitrary residue characteristic.

Given an irreducible representation $\sigma$ of ${\rm O}(V)$, the maximal $\sigma$-isotypic quotient of $\omega _{V,W,\psi }$ is of the form

\[ \sigma \boxtimes \Theta_{V,W,\psi}(\sigma) \]

for some representation $\Theta _{V,W,\psi }(\sigma )$ of ${\rm Mp}(W)$ (called the big theta lift of $\sigma$). Then $\Theta _{V,W,\psi }(\sigma )$ either is zero or has finite length. The maximal semi-simple quotient of $\Theta _{V,W,\psi }(\sigma )$ is denoted by $\theta _{V,W,\psi }(\sigma )$ (called the small theta lift of $\sigma$).

Similarly, if $\pi$ is an irreducible genuine representation of ${\rm Mp}(W)$, then one has its big theta lift $\Theta _{W,V,\psi }(\pi )$ and its small theta lift $\theta _{W,V,\psi }(\pi )$, which are representations of ${\rm O}(V)$.

By the Howe duality, each small theta lift is irreducible or zero [Reference WaldspurgerWal90; Reference Gan and TakedaGT16]. Gan and Savin [Reference Gan and SavinGS12, § 6] showed that:

1. (i) for $\pi \in {\rm Irr}({\rm Mp}(W))$, exactly one of $\theta _{W,V^+,\psi }(\pi )$ or $\theta _{W,V^-,\psi }(\pi )$ is nonzero;

2. (ii) given $\sigma \in {\rm Irr}({\rm SO}(V))$, with the extensions $\sigma ^+$ and $\sigma ^-$ to ${\rm O}(V)$, exactly one of $\Theta _{V,W,\psi }(\sigma ^+)$ or $\Theta _{V,W,\psi }(\sigma ^-)$ is nonzero.

Here ${\rm Irr}({\rm Mp}(W))$ is the set of equivalence classes of irreducible genuine representations of ${\rm Mp}(W)$, and $\sigma ^\pm$ denote the extensions such that $-1 \in {\rm O}(V)$ acts as $\pm 1$, respectively. Then Gan and Savin derived the following theorems [Reference Gan and SavinGS12, Theorems 1.1 and 1.3].

Theorem 6.1 There is a bijection

\[ \Theta_\psi : {\rm Irr}({\rm Mp}(W)) \longleftrightarrow {\rm Irr}({\rm SO}(V^+)) \sqcup {\rm Irr}({\rm SO}(V^-)), \]

given by the theta correspondence with respect to $\psi$.

See [Reference Gan and IchinoGI14, Appendix B] and [Reference Gan and IchinoGI16, Appendix A.7] for details on Plancherel measures.

By combining Theorem 6.2 with Theorem 5.1 and [Reference HanzerHan19], we obtain Theorem 4.1.

7.1 Representatives of a Weyl group element

Let $w \in {\rm W}(\hat {M}, {\rm Sp}_{2n}(\mathbb {C}))$ be a Weyl group element. We shall identify $w$ with elements in ${\rm W}(M, {\rm Mp}(W))$ and ${\rm W}(L, {\rm SO}(V))$ in a standard way, and take representatives $\widetilde {w}_P \in {\rm Mp}(W)$ and $\widetilde {w}_Q \in {\rm SO}(V)$ following the work of Langlands and Shelstad [Reference Langlands and ShelstadLS87] and Gan and Li [Reference Gan and LiGL18]. In this subsection we review the procedure; see [Reference Langlands and ShelstadLS87, § 2.1], [Reference ArthurArt13, § 2.3], or [Reference Gan and LiGL18, Definition 4.1] for details.

First, we realize the relative Weyl group ${\rm W}(\hat {M}, {\rm Sp}_{2n}(\mathbb {C}))$ in $\mathfrak {S}_{m} \ltimes (\mathbb {Z}/2\mathbb {Z})^{m}$ and identify it with relative Weyl groups ${\rm W}(\overline {M}, {\rm Sp}(W))$ and ${\rm W}(L, {\rm SO}(V))$. We can do this in a canonical way, because the Levi subgroups $\hat {M}$, $\overline {M}$, and $L$ are of the form

\[ {\rm GL}_{k_1} \times \cdots \times {\rm GL}_{k_m} \times G_- \]

over $\mathbb {C}$ or $F$, where $G_-$ is a semi-simple algebraic group.

Next, we take standard splittings $\textbf {spl}_{{\rm Sp}(W)}$ of ${\rm Sp}(W)$ and $\textbf {spl}_{{\rm SO}(V^+)}$ of ${\rm SO}(V^+)$ by

\[ \textbf{spl}_{{\rm Sp}(W)} = (B_W, T_W, \{X_{\alpha_i}\}_{i=1,\ldots,n}),\quad \textbf{spl}_{{\rm SO}(V^+)} = (B_V, T_V, \{X_{\beta_i}\}_{i=1,\ldots,n}), \]

where:

1. – $B_W$ and $B_V$ are, respectively, the Borel subgroups stabilizing the $F$-flags

   \[ Fy_1 \subset Fy_1 + Fy_2 \subset \cdots \subset Fy_1 + \cdots +Fy_n,\quad Fx_1 \subset Fx_1 + Fx_2 \subset \cdots \subset Fx_1 + \cdots +Fx_n; \]

2. – $T_W$ and $T_V$ are, respectively, their maximal tori which are diagonalized by the bases

   \[ \left\{ {y_1,\ldots,y_n,y_1^*,\ldots,y_n^*} \right\},\quad \left\{ {x_1,\ldots,x_n,x_0,x_1^*,\ldots,x_n^*} \right\}; \]

3. – $X_{\alpha _i}$ and $X_{\beta _i}$ are simple root vectors given by

   \begin{align*} X_{\alpha_i} : y_j &\mapsto \begin{cases} y_i & j=i+1, \\ 0 & \text{otherwise}, \end{cases}& y^*_j&\mapsto \begin{cases} -y^*_{i+1} & j=i, \\ 0 &\text{otherwise}, \end{cases}& \text{for }1\leq i\leq n-1,\\X_{\alpha_n} : y_j &\mapsto 0, & y^*_j&\mapsto \begin{cases} y_n & j=n, \\ 0 &\text{otherwise}, \end{cases}& \\X_{\beta_i} : x_j &\mapsto \begin{cases} x_i & j=i+1, \\ 0 &\text{otherwise}, \end{cases}& x^*_j&\mapsto \begin{cases} -x^*_{i+1} & j=i, \\ 0 &\text{otherwise}, \end{cases}& \text{for }1\leq i\leq n-1,\\X_{\beta_n} : x_j &\mapsto \begin{cases} 2x_n & j=0, \\ 0 &\text{otherwise}, \end{cases}& x^*_j&\mapsto \begin{cases} -x_0 & j=n, \\ 0 &\text{otherwise}. \end{cases}&\end{align*}

Then $M$ and $P$ (respectively $L$ and $Q$) are standard, in the sense that they contain $\widetilde {T_W}$ and $\widetilde {B_W}$ (respectively $T_V$ and $B_V$), respectively. Let $\Phi (T_W,{\rm Sp}(W))$ and $\Delta (B_W)=\{\alpha _1,\ldots ,\alpha _n\}$ denote the set of roots and the set of simple positive roots with the indices relative to the basis. Then we can see that the simple root vectors $X_{\alpha _i}$ do indeed correspond to $\alpha _i$. Let $X_{-\alpha _i}$ be the root vector for $-\alpha _i$ such that the Lie bracket $[X_{\alpha _i}, X_{-\alpha _i}]$ is the coroot for $\alpha _i$. Let us take $\Phi (T_V,{\rm SO}(V^+))$, $\Delta (B_V)=\{\beta _1,\ldots ,\beta _n \}$, and $X_{-\beta _i}$ similarly.

Now let us take representatives $\widetilde {w}_P$ and $\widetilde {w}_Q$ of $w$. First assume that $V=V^+$. Let $w_T$ and $w_T'$ denote the representatives of $w$ in the Weyl groups ${\rm W}(T_W,{\rm Sp}(W))$ and ${\rm W}(T_V, {\rm SO}(V^+))$ that stabilize the simple positive roots inside $\overline {M}$ and $L$, respectively. We shall write $w_\lambda$ for the reflection corresponding to a root $\lambda$. We then have Langlands–Shelstad representatives

\[ \widetilde{w}_P = \widetilde{w}_{\alpha_{(1)}} \cdots \widetilde{w}_{\alpha_{(\ell)}},\quad \widetilde{w}_Q = \widetilde{w}_{\beta_{(1)}} \cdots \widetilde{w}_{\beta_{(\ell)}}, \]

where $w_T = w_{\alpha _{(1)}}\cdots w_{\alpha _{(\ell )}}$ and $w_T'=w_{\beta _{(1)}}\cdots w_{\beta _{(\ell )}}$ are reduced decompositions of $w_T$ and $w_T'$ in ${\rm W}(T_W,{\rm Sp}(W))$ and ${\rm W}(T_V, {\rm SO}(V^+))$, respectively, and

\[ \widetilde{w}_{\alpha} = (\exp(X_\alpha)\exp(-X_{-\alpha})\exp(X_\alpha), 1 ),\quad \widetilde{w}_{\beta} = \exp(X_\beta)\exp(-X_{-\beta})\exp(X_\beta) \]

for any $\alpha \in \Delta (B_W)$ and $\beta \in \Delta (B_V)$.

In the case of $V=V^-$, the representative $\widetilde {w}_Q \in {\rm SO}(V^-)$ is defined to be the corresponding element via the canonical pure inner twist $(\xi , z) : {\rm SO}(V^+) \to {\rm SO}(V^-)$. The following lemma is obvious but important.

Lemma 7.1 Let $w=w_1 \cdots w_l$ be a reduced decomposition of $w$ as an element of the relative Weyl group ${\rm W}(\hat {M},{\rm Sp}_{2n}(\mathbb {C}))$. Then we have

\[ \widetilde{w}_P=\widetilde{w_1}_P \cdots \widetilde{w_l}_P \quad\text{and}\quad \widetilde{w}_Q=\widetilde{w_1}_Q \cdots \widetilde{w_l}_Q. \]

The representatives can be given more explicitly in the $m=1$ case. See Proposition 8.1 below.

7.2 Haar measures on the unipotent radicals

In this subsection we shall choose Haar measures on $N$ and $U$. We first define Haar measures $du$ on $U$ for $V=V^+$ and $d'n$ on $N$ with respect to the splittings $\textbf {spl}_{{\rm SO}(V^+)}$ and $\textbf {spl}_{{\rm Sp}(W)}$, following [Reference ArthurArt13, § 2.3] or [Reference Kaletha, Mínguez, Shin and WhiteKMSW14, § 2.2]. Here, $d'n$ is a Haar measure when we regard $N$ as the unipotent radical of a parabolic subgroup $\overline {P}$ of ${\rm Sp}(W)$. On the unipotent radical $N$ of the parabolic subgroup $P$ of ${\rm Mp}(W)$, we take a Haar measure $dn = |2|_F^{-k/2} d'n$.

Since the splittings are given explicitly, one can describe these measures explicitly. We will give an explicit definition of the measures in the case of $m=1$, i.e. $\mathbf{k}=(k)$, in § 8.5 below. The measures for $m=1$ give us the following descriptions of $du$ and $dn$.

For $1 \leq i\leq m$, put $n_i = n-(k_0+\cdots +k_{i-1})$. As in §§ 2.1 and 2.3, let $N^{(i)}$ and $U^{(i)}$ be the unipotent radicals of the maximal parabolic subgroups of ${\rm Sp}(W_{n_i})$ and ${\rm SO}(V_{n_i})$ stabilizing

\[ {{\rm span}}_F(y_{k_0+\cdots+k_{i-1}+1}, \ldots, y_{k_0+\cdots+k_i}),\quad {{\rm span}}_F(x_{k_0+\cdots+k_{i-1}+1}, \ldots, x_{k_0+\cdots+k_i}), \]

respectively. If we take Haar measures on each $N^{(i)}$ and $U^{(i)}$ as we will do on $N_k$ and $U_k$ in § 8.5 below, then the Haar measures $dn$ on $N$ and $du$ on U are the measures defined via the homeomorphisms

\begin{alignat*}{4} N^{(1)} \times \cdots \times N^{(m)} & \longrightarrow N, & \quad (n_1, \ldots, n_m) & \mapsto n_1 \cdots n_m,\\ U^{(1)} \times \cdots \times U^{(m)} & \longrightarrow U, & \quad (u_1, \ldots, u_m) & \mapsto u_1 \cdots u_m. \end{alignat*}

In the case of $V=V^-$, we define the Haar measure $du$ on $U$ by using the above homeomorphism.

7.3 Intertwining operators

Now we define intertwining operators. Let $\tau _i$ be irreducible tempered representations of ${\rm GL}_{k_i}$ on a vector space $\mathscr {V}_{\tau _i}$, for $i=1, \ldots , m$. For any $\textbf {s}=(s_1, \ldots , s_m) \in \mathbb {C}^m$, we realize the representation $\tau _{i, s_i}=\tau _i \otimes |\!\det \!|_F^{s_i}$ on $\mathscr {V}_{\tau _i}$ by setting $\tau _{i, s_i}(a)v=|\!\det a|_F^{s_i} \tau _i(a)v$ for $v \in \mathscr {V}_{\tau _i}$ and $a \in {\rm GL}_{k_i}$. Let $\pi _0$ be an irreducible genuine tempered representation of ${\rm Mp}(W_{n_0})$ on $\mathscr {V}_{\pi _0}$ and $\sigma _0$ an irreducible tempered representation of ${\rm SO}(V_{n_0})$ on $\mathscr {V}_{\sigma _0}$ such that $\pi _0$ and $\sigma _0$ correspond under the bijection $\Theta _\psi$. Put $\pi _{M,\textbf {s}} = \widetilde {\tau }_{1,s_1}\otimes \cdots \otimes \widetilde {\tau }_{m,s_m} \otimes \pi _0$ and $\sigma _{L,\textbf {s}} = \tau _{1,s_1}\otimes \cdots \otimes \tau _{m,s_m} \otimes \sigma _0$. In particular, we shall write $\pi _M=\pi _{M,0}$ and $\sigma _L=\sigma _{L,0}$.

The normalized parabolically induced representation ${\rm Ind}_P^{{\rm Mp}(W)}(\pi _{M,\textbf {s}})$ is realized on the space of $(\mathscr {V}_{\tau _1} \otimes \cdots \otimes \mathscr {V}_{\tau _m} \otimes \mathscr {V}_{\pi _0})$ -valued smooth functions $\mathscr {F}_\textbf {s}$ on ${\rm Mp}(W)$ such that

\[ \mathscr{F}_\textbf{s}(mng) =\delta_P(m)^{{1}/{2}} \pi_{M, \textbf{s}}(m) \mathscr{F}_\textbf{s}(g) \]

for any $m \in M$, $n \in N$, and $g \in {\rm Mp}(W)$, where $\delta _P$ is the modulus function. For $w \in {\rm W}(\hat {M}, {\rm Sp}_{2n}(\mathbb {C}))$ we define the unnormalized intertwining operator

\[ \mathcal{M} (\widetilde{w}_P, \pi_{M,\textbf{s}} ): {\rm Ind}_P^{{\rm Mp}(W)}(\pi_{M,\textbf{s}}) \to {\rm Ind}_P^{{\rm Mp}(W)}(w\pi_{M,\textbf{s}}) \]

by the meromorphic continuation of the integral

\[ \mathcal{M} (\widetilde{w}_P,\pi_{M,\textbf{s}} )\mathscr{F}_\textbf{s}(g) = \int_{(^{w}{N} \cap N) \backslash N} \mathscr{F}_\textbf{s}((\widetilde{w}_P)^{-1} ng)\,dn, \]

where $^{w} {N}=\widetilde {w}_P N \widetilde {w}_P^{-1}$ and $w\pi _{M,\textbf {s}}$ is the representation of $M$ on $\mathscr {V}_{\tau _1} \otimes \cdots \otimes \mathscr {V}_{\tau _m} \otimes \mathscr {V}_{\pi _0}$ given by

\[ w\pi_{M,\textbf{s}}(m) = \pi_{M,\textbf{s}}((\widetilde{w}_P)^{-1} m \widetilde{w}_P) \]

for $m \in M$. The integral above converges absolutely on some open set of $\mathbb {C}^m$ in $\textbf {s}$ and has a meromorphic continuation to $\textbf {s} \in \mathbb {C}^m$. The operator is well-defined for $\textbf {s} \in \mathbb {C}^m$ except at finite poles modulo $(2\pi i/\log q_F)\mathbb {Z}^m$. Similarly, we can define the unnormalized intertwining operator $\mathcal {M}(\widetilde {w}_Q, \sigma _{L,\textbf {s}})$ from ${\rm Ind}_Q^{{\rm SO}(V)}(\sigma _{L,\textbf {s}})$ to ${\rm Ind}_Q^{{\rm SO}(V)}(w\sigma _{L,\textbf {s}})$.

Before stating the definition of the normalized self-intertwining operators, we need to normalize the operators $\mathcal {M}(\widetilde {w}_P, \pi _{M,\textbf {s}})$ and $\mathcal {M}(\widetilde {w}_Q, \sigma _{L,\textbf {s}})$ so that they are holomorphic at $\textbf {s}=0$. Put $P^w = (\widetilde {w}_P)^{-1} P \widetilde {w}_P$ and $Q^w = (\widetilde {w}_Q)^{-1} Q \widetilde {w}_Q$, and let $\phi _1, \ldots , \phi _m, \phi _0$ be the $L$-parameters corresponding to $\tau _1, \ldots , \tau _m, \sigma _0$ via the LLC. Since $\sigma _0 = \Theta _\psi (\pi _0)$, the $L$-parameter for $\pi _0$ is also $\phi _0$. As in (3.1), put

\[ \phi = \phi_1 \oplus \cdots \oplus \phi_m \oplus \phi_0 \oplus \phi_m^\vee \oplus \cdots \oplus \phi_1^\vee \]

so that $\phi \in {\Phi }_{\rm temp}({\rm SO}_{2n+1}) = {\Phi }_{\rm temp}({\rm Mp}_{2n})$ and ${{\rm Im}}(\phi ) \subset \hat {M}$. We define the twist $\phi _\textbf {s}$ of $\phi$ by $\textbf {s}$ as follows:

\[ \phi_\textbf{s} = (\phi_1 \otimes |-|^{s_1}) \oplus \cdots \oplus (\phi_m \otimes |-|^{s_m}) \oplus \phi_0 \oplus (\phi_m^\vee \otimes |-|^{-s_m}) \oplus \cdots \oplus (\phi_1^\vee \otimes |-|^{-s_1}). \]

Then we define a normalizing factor

\[ r_{P^w|P}(\textbf{s},\phi, \psi) = \gamma(0, \rho_{P^w|P}^\vee \circ \phi_{\textbf{s}}, \psi)^{-1}, \]

where $\rho _{P^w|P}$ denotes the representation of $\hat {M}$ defined in [Reference ArthurArt13, pp. 80–81]. Similarly, define a normalizing factor $r_{Q^w|Q}(\textbf {s},\phi , \psi ) = \gamma (0, \rho _{Q^w|Q}^\vee \circ \phi _{\textbf {s}}, \psi )^{-1}$. The realization of ${\rm W}(\hat {M}, {\rm Sp}_{2n}(\mathbb {C}))$ as a subgroup of $\mathfrak {S}_{m} \ltimes (\mathbb {Z}/2\mathbb {Z})^{m}$ gives an expression

\[ w = \sigma_w \ltimes (d_i)_{i=1}^m. \]

Let $y : \mathbb {Z}/2\mathbb {Z} \to \{0,1\}$ be a map such that $y(2\mathbb {Z}) = 0$ and $y(1+2\mathbb {Z}) = 1$. We then define a representation $y(w, \phi _\textbf {s})$ of $\textit {WD}_F$ and a complex number $y(w, \textbf {s})$ by

\[ y(w, \phi_\textbf{s}) = \bigoplus_{i=1}^m y(d_i) \phi_i\otimes|-|^{s_i},\quad y(w, \textbf{s}) = \sum_{i=1}^m y(d_i) s_i. \]

Let us define normalized intertwining operators

\begin{align*} \mathcal{R}_P(w, \pi_{M, \textbf{s}}, \psi) &= \gamma_F(\psi)^{\dim y(w,\phi)} |2|_F^{2y(w,\textbf{s})} \gamma(\tfrac{1}{2}, y(w, \phi_\textbf{s}), \psi)^{-1} r_{P^w|P}(\textbf{s},\phi, \psi)^{-1} \mathcal{M} (\widetilde{w}_P,\pi_{M,\textbf{s}} ), \\ \mathcal{R}_Q(w, \sigma_{L, \textbf{s}}) &= \epsilon(V)^{\dim y(w, \phi)} r_{Q^w|Q}(\textbf{s},\phi, \psi)^{-1} \mathcal{M} (\widetilde{w}_Q,\sigma_{L,\textbf{s}} ). \end{align*}

It is known that $\mathcal {R}_Q(w, \sigma _{L, \textbf {s}})$ is independent of the choice of the additive character $\psi$ (see [Reference ArthurArt13, p. 83]). We can see that the intertwining operators can be defined at $\textbf {s}=0$.

We shall put

\[ \mathcal{R}_P(w, \pi_M, \psi) = \mathcal{R}_P(w, \pi_{M,0}, \psi),\quad \mathcal{R}_Q(w, \sigma_L) = \mathcal{R}_Q(w, \sigma_{L,0}). \]

Let us define the normalized self-intertwining operators. Assume that

\[ w \in {\rm W}_\phi(M,{\rm Mp}(W)) = {\rm W}_\phi(L,{\rm SO}(V)), \]

which is equivalent to $w \pi _M \cong \pi _M$ and $w \sigma _L \cong \sigma _L$. We take the unique Whittaker normalized isomorphism

\[ \mathcal{A}_w : \mathscr{V}_{\tau_1} \otimes \cdots \otimes \mathscr{V}_{\tau_m} \longrightarrow \mathscr{V}_{\tau_1} \otimes \cdots \otimes \mathscr{V}_{\tau_m} \]

and define the normalized self-intertwining operators

\begin{align*} R_P(w, \pi_M) &: {\rm Ind}_P^{{\rm Mp}(W)}(\pi_M) \longrightarrow {\rm Ind}_P^{{\rm Mp}(W)}(\pi_M),\\ R_Q(w, \sigma_L) &: {\rm Ind}_Q^{{\rm SO}(V)}(\sigma_L) \longrightarrow {\rm Ind}_Q^{{\rm SO}(V)}(\sigma_L) \end{align*}

as in [Reference Gan and IchinoGI16, p. 756].

7.4 Reduction

Since the LLC for ${\rm Mp}(W)$ is defined by using the theta correspondence, to prove Theorem 4.2 it suffices to consider the relation between the theta correspondence and the intertwining operators. The following proposition will be proved later.

Proposition 7.3 Put $\breve {\sigma }_L = \tau _1 \otimes \cdots \otimes \tau _m \otimes \sigma _0^\vee$. There exists a nonzero ${\rm SO}(V) \times {\rm Mp}(W)$-equivariant map

\[ \mathcal{T} : \omega_{V,W,\psi} \otimes {\rm Ind}_Q^{{\rm SO}(V)}(\breve{\sigma}_L) \longrightarrow {\rm Ind}_P^{{\rm Mp}(W)}(\pi_M) \]

such that:

1. (a) for any irreducible constituent $\sigma$ of ${\rm Ind}_Q^{{\rm SO}(V)}(\breve {\sigma }_L)$, the restriction of $\mathcal {T}$ to $\omega _{V,W,\psi } \otimes \sigma$ is nonzero;

2. (b) the diagram

   
   commutes.

Once the above proposition is proven, we have Theorem 4.2.

Proof. Suppose that Proposition 7.3 holds. Because any irreducible representation of an odd special orthogonal group is self-dual [Reference Mœglin, Vigneras and WaldspurgerMVW87, Ch. 4, § II.1], we have $\sigma _0^\vee \cong \sigma _0$. Now fix an isomorphism $\sigma_L \cong \breve{\sigma}_L$ and identify $\breve{\sigma}_L$ with $\sigma_L$.

Let $\pi \subset {\rm Ind}_P^{{\rm Mp}(W)}(\pi _M)$ be an irreducible tempered representation and put $\sigma = \Theta _\psi (\pi ) \subset {\rm Ind}_Q^{{\rm SO}(V)}(\sigma _L)$. Then, by Proposition 7.3 and the identification $\sigma _L \cong \breve {\sigma }_L$, there exists a nonzero ${\rm SO}(V) \times {\rm Mp}(W)$-equivariant map

\[ \mathcal{T} : \omega_{V,W,\psi} \otimes {\rm Ind}_Q^{{\rm SO}(V)}(\sigma_L) \longrightarrow {\rm Ind}_P^{{\rm Mp}(W)}(\pi_M) \]

such that its restriction to $\omega _{V,W,\psi } \otimes \sigma$ is nonzero and it satisfies the following commutative diagram:

By the Howe duality and the fact that $\sigma ^\vee \cong \sigma$, $\:\mathcal {T}$ sends $\omega _{V,W,\psi } \otimes \sigma$ to $\pi$. Therefore, $\mathcal {T}$ gives a nonzero ${\rm SO}(V) \times {\rm Mp}(W)$-equivariant map

\[ \mathcal{T}_{\sigma, \pi} : \omega_{V,W,\psi} \otimes \sigma \longrightarrow \pi \]

such that

(7.1)\begin{equation} R_P(w, \pi_M)\big|_\pi \circ \mathcal{T}_{\sigma, \pi} =\mathcal{T}_{\sigma, \pi} \circ \bigl(1_{\omega_{V,W,\psi}} \otimes R_Q(w, \sigma_L)\big|_\sigma\bigr). \end{equation}

Suppose that Hypothesis 5.2 holds. Then we have $R_Q(w, \sigma _L)|_\sigma = \eta (x_w)$ where $\eta = \iota (\sigma )$, and $x_w \in S_\phi ^\natural (L, {\rm SO}(V)) = S_\phi ^\natural (M, {\rm Mp}(W))$ is the image of $w$ under the natural map (3.2). Now the relation (7.1) shows that

\[ R_P(w, \pi_M)\big|_\pi \circ \mathcal{T}_{\sigma, \pi} =\eta(x_w) \mathcal{T}_{\sigma, \pi}. \]

Since $\mathcal {T}_{\sigma , \pi } \neq 0$ and $\pi$ is irreducible, we have $R_P(w, \pi _M)|_\pi = \eta (x_w)$. We also have $\eta = \iota _\psi (\pi )$ by the definition of the LLC for ${\rm Mp}(W)$. This completes the proof.

8.1 Maximal parabolic subgroups

We have described the parabolic subgroups of ${\rm Sp}(W)$, ${\rm Mp}(W)$, and ${\rm SO}(V)$ in §§ 2.1–2.3. Referring to [Reference AtobeAto18], we can describe their maximal parabolic subgroups more explicitl

Let $k$ be a positive integer, and put $n_0=n-k$. Put $Y= Y_k$ and $Y^*= Y^*_k$. We shall write an element in the symplectic group ${\rm Sp}(W)$ as a block matrix relative to the decomposition $W=Y \oplus W_{n_0} \oplus Y^*$. Following § 2.1 or 2.2, put $P=P_k$, $M=M_k$, and $N=N_k$ so that $\overline {P}=\overline {P_k}$ and $\overline {M}=\overline {M_k}$. Then we have

\begin{align*} \overline{M}&=\{ m(a)g_0 \mid a \in {\rm GL}(Y),\ g_0 \in {\rm Sp}(W_{n_0}) \}, \\ N&=\{ n^{\rm b}(b) n^{\rm c}(c) \mid b \in {\rm Hom}(W_{n_0},Y),\ c \in {\rm Sym}(Y^*,Y) \}, \end{align*}

where $m(a)$, $n^{\rm b}(b)$, $n^{\rm c}(c)$, and ${\rm Sym}(Y^*,Y)$ are defined as in [Reference AtobeAto18, § 2.4]. Recall that $P$ and $M$ are the double covers of $\overline {P}$ and $\overline {M}$, respectively. Note that the natural inclusion ${\rm Sp}(W_{n_0}) \subset {\rm Sp}(W)$ induces an inclusion ${\rm Mp}(W_{n_0}) \subset {\rm Mp}(W)$, $(g_0,\epsilon ) \mapsto (g_0,\epsilon )$. Put

\[ \rho_P=\frac{2n-k+1}{2}. \]

Assume that $k\leq r$. Put $X=X_k$ and $X^*=X^*_k$, and write an element in the special orthogonal group ${\rm SO}(V)$ as a block matrix relative to the decomposition $V=X\oplus V_{n_0} \oplus X^*$, as above. Put $Q=Q_k$, $L=L_k$, and $U=U_k$, following § 2.3. Then we have

\begin{align*} L &=\{l(a)h_0 \mid a \in {\rm GL}(X),\ h_0 \in {\rm SO}(V_{n_0})\}, \\ U &= \{ u = u^{\rm b}(b) u^{\rm c}(c) \mid b \in {\rm Hom}(V_{n_0},X),\ c \in {\rm Alt}(X^*,X) \}, \end{align*}

where $l(a)$, $u^{\rm b}(b)$, $u^{\rm c}(c)$, and ${\rm Alt}(X^*,X)$ are given in a similar way to [Reference AtobeAto18, § 2.4]. Put

\[ \rho_Q=\frac{2n-k}{2}. \]

8.2 Representatives of $w_M$ and $w_L$

Let $w_M$ (respectively $w_L$) be the nontrivial element of the relative Weyl group ${\rm W}(\overline {M}, {\rm Sp}(W))$ (respectively ${\rm W}(L, {\rm SO}(V))$). Note that ${\rm W}(\overline {M},{\rm Sp}(W)) \cong {\rm W}(L, {\rm SO}(V)) \cong \mathbb {Z}/2\mathbb {Z}$. In this subsection, we shall take representatives of $w_M$ and $w_L$, following Langlands and Shelstad (see § 7.1), and calculate them explicitly.

First, let us define $I_X \in {\rm Hom}(X^*,X)$ and $I_Y \in {\rm Hom}(Y^*,Y)$ by $I_X x_i^* = x_i$ and $I_Y y_i^* = y_i$. With respect to the bases, $I_X$ and $I_Y$ correspond to the identity matrix. Put

\[ J =\left(\begin{array}{cccc} & & & (-1)^{n+1} \\ & & (-1)^{n+2} & \\ & {\mathinner{\mkern2mu\raise1pt\hbox{.}\mkern2mu \raise4pt\hbox{.}\mkern2mu\raise7pt\hbox{.}\mkern1mu}} & & \\ (-1)^{n+k} & & & \end{array} \right) \in {\rm GL}_k. \]

Using the bases, we can identify ${\rm GL}(X)$ and ${\rm GL}(Y)$ with ${\rm GL}_k$ and regard $J$ as an element of ${\rm GL}(X)$ or ${\rm GL}(Y)$. Let us define elements $w_Y \in {\rm Sp}(W)$ and $w_X \in {\rm SO}(V)$ by

\[ w_Y = \left(\begin{array}{ccc} & & I_Y \\ & (-1)^k1_{W_{n_0}} & \\ -I_Y^{-1} & & \end{array}\right),\quad w_X = \left(\begin{array}{ccc} & & -I_X \\ & (-1)^k1_{V_{n_0}} & \\ -I_X^{-1} & & \end{array}\right). \]

We take the representatives $w_M' \in {\rm Mp}(W)$ and $w_L' \in {\rm SO}(V)$ of $w_M$ and $w_L$ defined by

\[ w_M' =\left((-1)^k\begin{pmatrix} & & -J I_Y \\ & 1_{W_{n_0}} & \\ J I_Y^{-1} & & \end{pmatrix}, \; \epsilon^\mathrm{LS}\right),\quad w_L' =(-1)^k\begin{pmatrix} & & J I_X \\ & 1_{V_{n_0}} & \\ J I_X^{-1} & & \end{pmatrix}, \]

respectively, where $\epsilon ^\mathrm{LS} = (-1,-1)_F^{k(k-1)/2}$.

Let $\widetilde {w}_P$ and $\widetilde {w}_Q$ denote Langlands and Shelstad's representatives (see [Reference Langlands and ShelstadLS87, § 2.1] and [Reference Gan and LiGL18, Definition 4.1]) of $w_M$ and $w_L$ with respect to the $F$-splittings $\textbf {spl}_{{\rm Sp}(W)}$ and $\textbf {spl}_{{\rm SO}(V^+)}$. Then we have the following proposition.

Proposition 8.1 We have

\begin{align*} \widetilde{w}_P &= w'_M, \\ \widetilde{w}_Q &= w'_L. \end{align*}

The proof is similar to that of [Reference Gan and IchinoGI16, Lemma 7.2]. Also, as pointed out in [Reference Gan and IchinoGI16, p. 755], in the $V = V^-$ case one can see that $w_L'$ corresponds to $\widetilde {w}_Q \in {\rm SO}(V^+)$ via the canonical pure inner twist. However, we shall give a proof of the first assertion in § 8.4, since the calculation of $\widetilde {w}_P$ is too complicated because we have to consider Ranga Rao's 2-cocycle.

8.3 Ranga Rao's 2-cocycle

Before proving Proposition 8.1, we introduce some notation and review Ranga Rao's x-function and normalized cocycle [Reference Ranga RaoRan93].

For three nonnegative integers $r, s, t \in \mathbb {Z}_{\geq 0}$, define $\iota ^{{\rm GL}}_{r,s,t}$ to be an embedding of ${\rm GL}_s$ into ${\rm GL}_{r+s+t}$ by

\[ A \mapsto \left( \begin{array}{ccc} 1_r & & \\ & A & \\ & & 1_t \end{array} \right). \]

For $a \in {\rm GL}(Y_n)$, we write $m_n(a)$ for an element $\bigl (\begin {smallmatrix}a & \\ & (a^*)^{-1} \end {smallmatrix}\bigr )$ of a Levi subgroup $\overline {M_n}$ of the Siegel parabolic subgroup $\overline {P_n}$. For any subset $S \subset \left\{ {1,\ldots ,n } \right\}$, define $\sigma _S$ and $a_S$ by

\[ \sigma_S \cdot y_i=\begin{cases} y_i^* & \text{if } i\in S, \\ y_i & \text{if }i\notin S, \end{cases} \quad\sigma_S \cdot y_i^*=\begin{cases} -y_i & \text{if } i\in S, \\ y_i^* & \text{if }i\notin S \end{cases} \]

and

\[ a_S \cdot y_i=\begin{cases} -y_i & \text{if } i\in S, \\ y_i & \text{if }i\notin S, \end{cases} \quad a_S \cdot y_i^*=\begin{cases} -y_i^* & \text{if } i\in S, \\ y_i^* & \text{if }i\notin S. \end{cases} \]

When $S$ is a singleton $\{i\}$, we shall write $\sigma _i = \sigma _{\{i\}}$ for simplicity.

Next, we review the notions of Ranga Rao's $x$-function and normalized cocycle. We have ${\rm Sp}(W) = \cup _S \overline {P_n} \sigma _S \overline {P_n}$, where the disjoint union runs over all subsets $S \subset \{1, \ldots , n\}$, and Ranga Rao's $x$-function is defined by

\[ x(p_1 \sigma_S p_2) = \det(p_1p_2|_{Y_n}) ({\rm mod} (F^\times)^2 ), \quad p_1, p_2 \in \overline{P_n}, \ S \subset \left\{ {1, \ldots, n} \right\}. \]

This is well-defined [Reference Ranga RaoRan93, Lemma 5.1]. Then, let $c(-,-)$ denote Ranga Rao's normalized cocycle, which is a 2-cocycle on ${\rm Sp}(W)$ valued in $\{\pm 1\}$. The precise definition of $c(-,-)$ is omitted here, but we list several of its properties. See [Reference Ranga RaoRan93, § 5] or [Reference SzpruchSzp07, § 2] for details.

Proposition 8.2 Let $p, p' \in \overline {P_n}$, $S, S' \subset \left\{ {1, \ldots , n} \right\}$, and $g, g' \in {\rm Sp}(W)$. Put $j = |S \cap S'|$. Then

\begin{align*} c(\sigma_S, \sigma_{S'}) &= (-1,-1)_F^{{j(j+1)}/{2}}, \\ c(pg, g'p') &= c(g, g') (x(g), x(p))_F (x(g'), x(p'))_F (x(p), x(p'))_F (x(gg'), x(pp'))_F, \\ c(g,p) &= c(p,g) = (x(p), x(g))_F. \end{align*}

Moreover, if $gg' = g'g$, then

\[ c(g,g') = c(g',g). \]

8.4 Proof of Proposition 8.1

Now we begin the proof of the first assertion of Proposition 8.1.

Proof. First, we need a certain representative of $w_M$ in ${\rm W}(T_W,{\rm Sp}(W))$. Take the representative $w_T \in {\rm W}(T_W,{\rm Sp}(W))$ of $w_M$ such that $w_T$ maps the positive roots inside $\overline {M}$ into the positive roots inside $\overline {M}$ and the positive roots outside $\overline {M}$ into the negative roots (not necessarily outside $\overline {M}$). Let $s_i \in {\rm W}(T_W,\,{\rm Sp}(W))$ be the simple reflection corresponding to $\alpha _i \in \Delta (B_W)$, and put

\begin{align*} q_i &= s_{k-1}s_{k-2}\cdots s_{i+1}s_i \quad\text{for } 1\leq i\leq k-1,\\ r_i &= s_is_{i+1}\cdots s_{n-1}s_ns_{n-1}\cdots s_{i+1}s_i \quad\text{for } 1\leq i\leq n. \end{align*}

Then

\[ w_T = r_kq_1r_kq_2r_k\cdots q_{k-2}r_kq_{k-1}r_k \]

gives a reduced decomposition of $w_T$.

Second, let us consider the representative $\widetilde {w}_{\overline {P}}$ of $w_T$ in the symplectic group ${\rm Sp}(W)$, following Langlands and Shelstad [Reference Langlands and ShelstadLS87]. Put

\begin{align*} \omega_i &= \exp(X_{\alpha_i})\exp(-X_{-\alpha_i})\exp(X_{\alpha_i})\quad \text{for }1\leq i\leq n, \\ u_i &= \omega_{k-1}\cdots \omega_{i+1}\omega_i \quad\text{for }1\leq i\leq k-1, \\ v_i &= \omega_i\cdots \omega_{n-1}\omega_n\omega_{n-1}\cdots \omega_i \quad\text{for } 1\leq i\leq n, \end{align*}

which are representatives of $s_i$, $q_i$, and $r_i$, respectively. Then, by [Reference Langlands and ShelstadLS87, § 2.1], we have the representative $\widetilde {w}_{\overline {P}}$ in ${\rm Sp}(W)$:

\[ \widetilde{w}_{\overline{P}} = v_k u_1v_k u_2\cdots v_k u_{k-1}v_k. \]

In addition, put

\[ z_i = \omega_{k-1}^{-1} \cdots \omega_{i+1}^{-1} \omega_i^{-1} \quad \text{for } 1\leq i\leq k-1. \]

Then we obtain that $v_k u_i = z_iv_i$ for $1\leq i\leq k-1$. Moreover, one can calculate $v_i$ and $z_j$ by descending induction on $i=n,\ldots ,1$ and $j=k-1,\ldots ,1$ to obtain

\begin{align*} v_i &= a_{\{i+1, \ldots, n\}} \sigma_i^{2(n-i-1)+1}, \\ z_j &= m_n\bigl(\iota^{{\rm GL}}_{j-1, k-j+1, n-k}(\kappa_{k-j+1})\bigr), \end{align*}

where

\[ \kappa_l =\left(\begin{array}{cccc} 0 & -1 & & \\ & & \ddots & \\ & & & -1\\ 1 & & & 0\\ \end{array}\right) \in {\rm GL}_l. \]

A straightforward calculation then shows that $v_iz_j = z_jv_i$ for $1 \leq i < j \leq k-1$. This implies that $\widetilde {w}_{\overline {P}}=z_1\cdots z_{k-1}v_1\cdots v_k$.

Finally, let us take their representatives in ${\rm Mp}(W)$ as follows. Put

\begin{align*} \widetilde{\omega}_i &= \widetilde{w}_{\alpha_i} = (\omega_i,1) \quad\text{for } 1\leq i\leq n, \\ \widetilde{u}_i &= \widetilde{\omega}_{k-1}\cdots\widetilde{\omega}_{i+1}\widetilde{\omega}_i \quad\text{for } 1\leq i\leq k-1, \\ \widetilde{v}_i &=\widetilde{\omega}_i\cdots\widetilde{\omega}_{n-1}\widetilde{\omega}_n\widetilde{\omega}_{n-1}\cdots\widetilde{\omega}_i \quad\text{for } 1\leq i\leq n, \end{align*}

and

\[ \widetilde{z}_i = \widetilde{\omega}_{k-1}^{-1} \cdots \widetilde{\omega}_{i+1}^{-1} \widetilde{\omega}_i^{-1} \quad\text{for } 1\leq i\leq k-1. \]

Then the required element $\widetilde {w}_P$ can be expressed as

\[ \widetilde{w}_P = \widetilde{v}_k\widetilde{u}_1\widetilde{v}_k\widetilde{u}_2\cdots\widetilde{v}_k\widetilde{u}_{k-1}\widetilde{v}_k. \]

We have $\widetilde {v}_k\widetilde {u}_i=\widetilde {z}_i\widetilde {v}_i$ for $1\leq i\leq k-1$. Also, for $1\leq i < j\leq k-1$ we have $\widetilde {v}_i\widetilde {z}_j = \widetilde {z}_j \widetilde {v}_i$ because $v_iz_j = z_jv_i$. Therefore,

\begin{align*} \widetilde{w}_P&=\widetilde{z}_1\widetilde{v}_1\widetilde{z}_2\widetilde{v}_2\cdots\widetilde{z}_{k-1}\widetilde{v}_{k-1}\widetilde{v}_k \\ &=\widetilde{z}_1\widetilde{z}_2\cdots\widetilde{z}_{k-1}\widetilde{v}_1\widetilde{v}_2\cdots\widetilde{v}_{k-1}\widetilde{v}_k. \end{align*}

Since $\omega _1,\ldots , \omega _{n-1}$ are elements of the Siegel parabolic subgroup $\overline {P_n}$ and have determinant 1 on $Y_n$, one has

\begin{align*} \widetilde{v}_i &= (v_i,1) \quad \text{for } 1\leq i \leq k,\\ \widetilde{z}_j &= (z_j,1) \quad \text{for } 1 \leq j \leq k-1. \end{align*}

Now let us compute $\widetilde {z}_1\widetilde {z}_2\cdots \widetilde {z}_{k-1}$, $\widetilde {v}_1\widetilde {v}_2\cdots \widetilde {v}_{k-1}\widetilde {v}_k$, and $\widetilde {w}_P$. First, we consider $\widetilde {z}_1\widetilde {z}_2\cdots \widetilde {z}_{k-1}$. Since each $z_j$ belongs to the Siegel parabolic subgroup and has determinant $1$ on $Y_n$, we have $\widetilde {z}_1\cdots \widetilde {z}_{k-1}=(z_1\cdots z_{k-1},1)$. Additionally, by calculating its action on the basis $\{y_1,\ldots ,y_n,y_1^*,\ldots ,y_n^* \}$, we can compute the product $z_1\cdots z_{k-1}$:

\[ z_1\cdots z_{k-1} = m_n\bigl(\iota^{{\rm GL}}_{0,k,n-k}((-1)^{n+k}J)\bigr). \]

Second, by descending induction, we can compute $\widetilde {v}_i \cdots \widetilde {v}_k$ for $i=k,\ldots ,1$. Note that Ranga Rao's normalized cocycle may not be trivial. By descending induction, one has

\[ v_i\cdots v_k = \sigma_{\{i,\ldots,k\}}p'_i, \]

where

\[ p_i' = (a_{\{i, \ldots, k\}})^{n-i+1} (a_{\{k+1, \ldots, n\}})^{k-i+1}. \]

We also have

\[ v_i = p_i\sigma_i, \]

where

\[ p_i = (a_{\{i\}})^{n-i+1} a_{\{i+1, \ldots, n\}}. \]

Since $p_i'$ and $p_i$ are elements of the Siegel parabolic subgroup,

\begin{align*} c(v_i,v_{i+1}\cdots v_k) &=c(p_i\sigma_{i},\sigma_{\{i+1,\ldots,k\}}p_{i+1}') \\ &=(x(p_i),x(p_{i+1}'))_F \\ &=\bigl((-1)^{(n-i+1)+(n-i)},(-1)^{(n-i)(k-i)+(k-i)(n-k)}\bigr)_F\\ &=(-1,-1)_F^{k+i}. \end{align*}

Hence,

\begin{align*} \widetilde{v}_1\cdots\widetilde{v}_k &=(v_1\cdots v_k, \; \prod_{i=1}^{k-1} c(v_i,v_{i+1}\cdots v_k)) \\ &= ( (\sigma_{\{1, \ldots, k\}})^{2n+1} (a_{\{k+1, \ldots, n\}})^k, (-1,-1)_F^{{k(k-1)}/{2}} ). \end{align*}

Finally, since $z_1\cdots z_{k-1}$ belongs to the Siegel parabolic subgroup and has determinant $1$ on $Y_n$, we have $\widetilde {w}_P= (\widetilde {w}_{\overline {P}}, \epsilon ^\mathrm{LS} )$ and $\widetilde {w}_{\overline {P}}= z_1\cdots z_{k-1}v_1\cdots v_k = (-1)^km_n(\iota ^{{\rm GL}}_{0,k,n-k}(J)) \sigma _{\{1, \ldots , k\}}$. This proves the first assertion of the proposition.

8.5 Haar measures

In order to study the intertwining operators in more detail, or to describe some explicit formulas for the Weil representations, we need to take Haar measures appropriately and explicitly. Put

\begin{align*} e &=x_1\otimes y^*_1+\cdots+x_k\otimes y_k^* \in X\otimes Y^*, \\ e^* &=x^*_1\otimes y_1+\cdots+x^*_k\otimes y_k \in X^*\otimes Y, \\ e^{**}&=x^*_1\otimes y^*_1+\cdots+x^*_k\otimes y^*_k \in X^*\otimes Y^*. \end{align*}

These vectors belong to the symplectic space $\mathbb {W}=V \otimes _F W$.

Let us define measures on each of the groups and vector spaces.

1. (i) Take the self-dual Haar measure $d_{{\rm M}_k}x$ on ${\rm M}_k(F)$ with respect to the pairing

   \[ {\rm M}_k(F) \times {\rm M}_k(F) \ni (x,y)\mapsto \psi({\rm tr}(xy)) \in \mathbb{C}^1. \]
   In particular, write $d_\psi x$ when $k=1$.

2. (ii) Take the Haar measure $dx$ on ${\rm GL}_k(F)$ defined by $dx = |\!\det x|_F^{-k} d_{{\rm M}_k}x$, and transfer it to ${\rm GL}(Y)$ and ${\rm GL}(X)$ via the identification.

3. (iii) Define the self-dual Haar measures on $V\otimes Y^*$, $X^*\otimes Y$, $X\otimes Y^*$, $V_{n_0}\otimes Y^*$, $X^*\otimes W_{n_0}$, ${\rm Hom}(V_{n_0},X)$, ${\rm Hom}(W_{n_0},Y)$, ${\rm Hom}(X,X)$, and ${\rm Hom}(Y,Y)$ in a similar way to [Reference Gan and IchinoGI16, § 7.2].

4. (iv) Take the self-dual Haar measures on ${\rm Alt}(X^*,X)$ and ${\rm Sym}(Y^*,Y)$ with respect to the pairings

   \begin{align*} {\rm Alt}(X^*,X) \times {\rm Alt}(X^*,X) \ni (c,c') &\mapsto \psi(\langle I_Yce^{**}, I_X^{-1} c'e^{**} \rangle) \in \mathbb{C}^1,\\ {\rm Sym}(Y^*,Y) \times {\rm Sym}(Y^*,Y) \ni (c, c') &\mapsto \psi(\langle I_Xce^{**}, I_Y^{-1} c'e^{**} \rangle) \in \mathbb{C}^1, \end{align*}
   respectively.

5. (v) Take the Haar measures $du$ on $U$ for $u=u^{\rm b}(b)u^{\rm c}(c)$ and $dn$ on $N$ for $n=n^{\rm b}(b)n^{\rm c}(c)$ as follows:

   \begin{align*} du&=|2|_F^{-{k}/{2}}\,db \cdot |2|_F^{-{k(k-1)}/{4}}\,dc,\quad b\in{\rm Hom}(V_{n_0},X),\ c\in{\rm Alt}(X^*,X), \\ dn&=|2|_F^{-{k}/{2}}\,db \cdot |2|_F^{-{k(k-1)}/{4}}\,dc,\quad b\in{\rm Hom}(W_{n_0},Y),\ c\in{\rm Sym}(Y^*,Y). \end{align*}

6. (vi) Let us take measures on $Q$ and $\overline {P}$. For $q=lu \in Q=LU$ and $p=mn \in \overline {P}=\overline {M} N$, define

   \[ dq=dl\,du, \quad dp=dm\,dn. \]
   We have the modulus function $\delta _Q(l(a)h_0)=|\!\det a|_F^{2\rho _Q}$ for $a\in {\rm GL}(X)$ and $h_0\in {\rm SO}(V_{n_0})$ and the modulus function $\delta _P(m(a)g_0)=|\!\det a|_F^{2\rho _P}$ for $a\in {\rm GL}(Y)$ and $g_0\in {\rm Sp}(W_{n_0})$.

One can then check that the measures $du$ on $U$ and $dn$ on $N$ coincide with the Haar measures that we took in § 7.2 by using the splittings $\textbf {spl}_{{\rm SO}(V^+)}$ and $\textbf {spl}_{{\rm Sp}(W)}$, respectively (see [Reference AtobeAto18, § 6.3] for explicit calculations).

8.6 Big symplectic spaces and a mixed model

In this subsection we shall take a mixed model, which is a realization of the Weil representation, following Gan and Ichino [Reference Gan and IchinoGI16, § 7.4].

Put $\mathbb {W}_0=V\otimes W_{n_0}\subset \mathbb {W}$ and $\mathbb {W}_{00}=V_{n_0}\otimes W_{n_0} \subset \mathbb {W}_0 \subset \mathbb {W}$. These are symplectic subspaces of $\mathbb {W}$. Fix a polarization $W_{n_0}=W_{01}\oplus W_{02}$, where $W_{01} = {{\rm span}}_F(y_{k+1}, \ldots , y_n)$ and $W_{02} = {{\rm span}}_F(y^*_{k+1}, \ldots , y_n^*)$. We have the following natural complete polarizations of $\mathbb {W}$, $\mathbb {W}_0$, and $\mathbb {W}_{00}$:

\begin{align*} \mathbb{W} &= (V\otimes Y_n) \oplus (V\otimes Y^*_n), \\ \mathbb{W}_0 &= (V\otimes W_{01}) \oplus (V\otimes W_{02}), \\ \mathbb{W}_{00}&= (V_{n_0}\otimes W_{01}) \oplus (V_{n_0}\otimes W_{02}). \end{align*}

Let $\omega$, $\omega _{0}$, and $\omega _{00}$ be the realizations of the Weil representations $\omega _{V,W,\psi }$, $\omega _{V, W_{n_0}, \psi }$, and $\omega _{V_{n_0}, W_{n_0}, \psi }$ of ${\rm O}(V)\times {\rm Mp}(W)$, ${\rm O}(V)\times {\rm Mp}(W_{n_0})$, and ${\rm O}(V_{n_0})\times {\rm Mp}(W_{n_0})$, respectively, on a mixed Schrödinger model

\begin{align*} \mathcal{S}_{00}&=\mathcal{S}(V_{n_0}\otimes W_{02}), \\ \mathcal{S}_0&= \mathcal{S}(X^*\otimes W_{0})\otimes \mathcal{S}_{00}, \\ \mathcal{S}&= \mathcal{S}(V\otimes Y^*)\otimes\mathcal{S}_0, \end{align*}

as in [Reference Gan and IchinoGI16, § 7.4] or [Reference AtobeAto18, § 6.2]. We construct these models by using the following elements:

1. – the ordinary Schrödinger models

   \[ (\omega^{{\rm or}}, \mathcal{S}^{{\rm or}}=\mathcal{S}(V\otimes(Y^*\oplus W_{02}))),\quad (\omega_0^{{\rm or}}, \mathcal{S}_0^{{\rm or}}=\mathcal{S}(V\otimes W_{02})),\quad (\omega_{00}^{{\rm or}}, \mathcal{S}_{00}^{{\rm or}}=\mathcal{S}(V_{n_0}\otimes W_{02})) \]
   of $\omega _{V,W,\psi }$, $\omega _{V, W_{n_0}, \psi }$, and $\omega _{V_{n_0}, W_{n_0}, \psi }$, respectively;

2. – canonical linear isomorphisms

   \begin{align*} \mathcal{S}(V\otimes (Y^*\oplus W_{02})) &\cong \mathcal{S}(V\otimes Y^*)\otimes \mathcal{S}(V\otimes W_{02}), \\ \mathcal{S}(V\otimes W_{02}) &\cong \mathcal{S}((X\oplus X^*)\otimes W_{02})\otimes \mathcal{S}(V_{n_0}\otimes W_{02}); \end{align*}

3. – an isomorphism given by the partial inverse Fourier transform

   \[ \mathcal{S}((X\oplus X^*)\otimes W_{02}) \to \mathcal{S}(X^*\otimes W_{0}), \quad \varphi \mapsto \hat{\varphi} \]
   defined by
   \[ \hat{\varphi} \begin{pmatrix}x_1\\x_2 \end{pmatrix} =\int_{y\in X\otimes W_{02}} \varphi \begin{pmatrix}y\\x_2 \end{pmatrix}\psi(-\langle x_1,y \rangle)\,dy \quad\text{for } x_1\in X^*\otimes W_{01},\ x_2\in X^*\otimes W_{02}, \]
   where the Haar measure $dy$ on $X \otimes W_{02}$ is defined by
   \[ dy = \prod_{\substack{1\leq i \leq k \\ k+1\leq j \leq n}} d_\psi c_{i,j} \quad \text{for } y = \sum_{\substack{1\leq i \leq k \\ k+1\leq j \leq n}} c_{i,j} x_i \otimes y_j^* \in X \otimes W_{02}. \]

Let $\mathscr {H}_0 = \mathbb {W}_0 \oplus F$ and $\mathscr {H}_{00} = \mathbb {W}_{00} \oplus F$ be the Heisenberg groups. Let $\rho _0$ and $\rho _{00}$ be their Heisenberg representations associated with the Weil representations $(\omega _0, \mathcal {S}_0)$ and $(\omega _{00}, \mathcal {S}_{00})$, respectively. We regard ${\rm Sp}(W) = {\rm Sp}(W)\times \{1\} \subset {\rm Sp}(W)\times \{\pm 1\} = {\rm Mp}(W)$ as sets. Referring to [Reference Ranga RaoRan93] or [Reference KudlaKud94, Theorem 3.1], we obtain some explicit formulas for the Weil representations.

For $\varphi \in \mathcal {S}$ and $x\in V\otimes Y^*$,

\begin{align*} [\omega(h)\varphi](x) &=\omega_0(h)\varphi(h^{-1}x), \quad h\in {\rm SO}(V), \\ [\omega(m(a))\varphi](x) &=\gamma_F(\det a,\psi)^{-1}|\!\det a|_F^{({2n+1})/{2}}\varphi(a^*x), \quad a\in {\rm GL}(Y), \\ [\omega(g_0)\varphi](x)&=\omega_0(g_0)\varphi(x),\quad g_0\in {\rm Sp}(W_{n_0}), \\ [\omega(n^{{\rm b}}(b))\varphi](x) &=\rho_0((b^*x,0))\varphi(x), \quad b\in{\rm Hom}(W_{n_0},Y), \\ [\omega(n^{\rm c}(c))\varphi](x) &=\psi (\tfrac{1}{2}\langle n^{\rm c}(c)x,x\rangle )\varphi(x), \quad c\in{\rm Sym}(Y^*,Y),\\ [\omega(w_Y^{-1})\varphi](x) &=\gamma_F(\psi\circ q_V)^{-k}\omega_0((-1_{W_{n_0}})^k)\int_{Y^*\otimes V}\psi(\langle x', I_Yx\rangle)\varphi(x')\,dx'. \end{align*}

For $\varphi _0\in \mathcal {S}_0=\mathcal {S}(X^*\otimes W_{n_0})\otimes \mathcal {S}_{00}$ and $\ y\in X^*\otimes W_{n_0}$,

\begin{align*} [\omega_0(g_0)\varphi_0](y) &=\omega_{00}(g_0)\varphi_0(g_0^{-1}y), \quad g_0 \in {\rm Sp}(W_{n_0}), \\ [\omega_0(l(a))\varphi_0](y) &=|\!\det a|_F^{n-k}\varphi_0(a^*y),\quad a\in {\rm GL}(X), \\ [\omega_0(h_0)\varphi_0](y) &=\omega_{00}(h_0)\varphi_0(y), \quad h_0 \in {\rm SO}(V_{n_0}), \\ [\omega_0(u^{\rm b}(b))\varphi_0](y) &=\rho_{00}((b^*y,0))\varphi_0(y), \quad b\in{\rm Hom}(V_{n_0},X), \\ [\omega_0(u^{\rm c}(c))\varphi_0](y) &= \psi (\tfrac{1}{2}\langle u^{\rm c}(c)y,y\rangle )\varphi_0(y), \quad c\in{\rm Alt}(X^*,X), \\ [\omega_0(w_X)\varphi_0](y) &=\omega_{00}((-1_{V_{n_0}})^k) \int_{X^*\otimes W_{n_0}}\psi(-\langle y',I_X y\rangle)\varphi_0(y')\,dy'. \end{align*}

For $\varphi _{00}\in \mathcal {S}_{00}=\mathcal {S}(V_{n_0}\otimes W_{02})$ and $x\in V_{n_0}\otimes W_{02}$,

\begin{align*} [\omega_{00}((-1_{W_{n_0}})^k)\varphi_{00}] (x) &=\gamma_F((-1)^{k(n-k)},\psi)^{-1}\varphi_{00}((-1)^kx), \\ [\omega_{00}((-1_{V_{n_0}})^k)\varphi_{00}](x) &=\varphi_{00}((-1)^kx). \end{align*}

8.7 Gan and Ichino's equivariant maps

Next, we construct equivariant maps which realize the theta correspondence. Put

\begin{align*} f_\mathcal{S}(\varphi)(gh) &=[[\omega(gh)\varphi](e)](0), \\ \hat{f}_\mathcal{S}(\varphi)(gh) &=\biggl[\int_{X\otimes Y^*}[\omega(gh)\varphi](x)\psi(-\langle e^*,x\rangle)\,dx\biggr](0) \end{align*}

for $\varphi \in \mathcal {S} =\mathcal {S}(V\otimes Y^*) \otimes \mathcal {S}(X^*\otimes W_{n_0}) \otimes \mathcal {S}_{00}$, $g \in {\rm Mp}(W)$, and $h \in {\rm O}(V)$. If $f=f_\mathcal {S}(\varphi )$ or $\hat {f}_\mathcal {S}(\varphi )$, then by the explicit formulas for the mixed Schrödinger model, we have

(8.1)\begin{align} f(nugh)&=f(gh),\quad n\in N, \:u\in U, \nonumber\\ f(g_0h_0gh)&=\omega_{00}(g_0h_0)f(gh),\quad g_0 \in {\rm Sp}(W_{n_0}),\: h_0 \in {\rm O}(V), \nonumber\\ f(m(a)l(a)gh)&=\gamma_F(\det a,\psi)^{-1}|\!\det a|_F^{\rho_Q+\rho_P}f(gh), \quad a\in {\rm GL}_k(F)\cong {\rm GL}(X)\cong {\rm GL}(Y), \end{align}

for any $g\in {\rm Mp}(W)$ and $h\in {\rm O}(V)$. In the rest of this section we shall drop the subscript $\mathcal {S}$ for simplicity.

In this subsection, we shall write $\tau =\tau _1$ and assume that $\sigma _0$ and $\pi _0$ may be direct sums of irreducible tempered representations, whose summands have the same $L$-parameter $\phi _0$ and correspond bijectively via $\Theta _\psi$. For $\rho =\tau$, $\pi _0,$ or $\sigma _0$, let $(\rho ^\vee , \mathscr {V}_{\rho ^\vee })$ be the contragredient representation of $(\rho , \mathscr {V}_\rho )$ and $\langle -,- \rangle$ the invariant non-degenerate bilinear form on $\mathscr {V}_\rho \times \mathscr {V}_{\rho ^\vee }$. We fix a nonzero ${\rm Mp}(W_{n_0}) \times {\rm SO}(V_{n_0})$-equivariant map

(8.2)\begin{equation} \mathcal{T}_{00} : \omega_{00} \otimes \sigma_0^\vee \longrightarrow \pi_0. \end{equation}

For any $\varphi \in \mathcal {S}$, $\mathscr {F}_s\in {\rm Ind}_Q^{{\rm SO}(V)}(\tau _s \otimes \sigma _0^\vee )$, $g\in {\rm Mp}(W)$, $\check {v} \in \mathscr {V}_{\tau ^\vee }$, and $\check {v}_0 \in \mathscr {V}_{\pi _0^\vee }$, put

\[ I(s,\varphi\otimes \mathscr{F}_s, \check{v}\otimes\check{v}_0, g) =\frac{1}{L(s+\tfrac{1}{2},\tau)} \int_{U{\rm SO}(V_{n_0})\backslash {\rm SO}(V)} \langle \mathcal{T}_{00}(\hat{f}(\varphi)(gh)\otimes \langle \mathscr{F}_s(h), \check{v} \rangle), \check{v}_0 \rangle \, dh \]

if the right-hand side converges absolutely. Here, $s \in \mathbb {C}$ is a complex variable.

When $s=0$ the assignment $\varphi \otimes \mathscr {F} \mapsto \mathcal {T}_0( \varphi \otimes \mathscr {F})$ gives an ${\rm Mp}(W) \times {\rm SO}(V)$-equivariant map $\omega \otimes {\rm Ind}_Q^{{\rm SO}(V)}(\tau \otimes \sigma _0^\vee ) \to {\rm Ind}_P^{{\rm Mp}(W)}(\widetilde {\tau }\otimes \pi _0)$. We shall write $\mathcal {T}(k, \mathcal {T}_{00})$ for this map.

Now, we note the functorialities of the equivariant map $\mathcal {T}(k,\mathcal {T}_{00})$ here. We have the following two lemmas, which easily follow from the definition of $\mathcal {T}(k,\mathcal {T}_{00})$.

Lemma 8.4 Let $(\tau ', \mathscr {V}_{\tau '})$ be a representation of ${\rm GL}_k$ that is isomorphic to $\tau$, and let $A : (\tau , \mathscr {V}_\tau ) \to (\tau ', \mathscr {V}_{\tau '})$ be an isomorphism of representations of ${\rm GL}_k$. Then the diagram

commutes. Here, ${\rm Ind}(A)$ denotes an operator defined by $[{\rm Ind}(A)\mathscr {F}](x) = A(\mathscr {F}(x))$.

Lemma 8.5 Let $(\sigma _0', \mathscr {V}_{\sigma _0'})$ (respectively $(\pi _0', \mathscr {V}_{\pi _0'})$) be a representation of ${\rm SO}(V_{n_0})$ (respectively ${\rm Mp}(W_{n_0})$) that is isomorphic to $\sigma _0$ (respectively $\pi _0$), and let $B : (\sigma _0^\vee , \mathscr {V}_{\sigma _0^\vee }) \to ({\sigma _0'}^\vee , \mathscr {V}_{{\sigma _0'}^\vee })$ (respectively $C : (\pi _0, \mathscr {V}_{\pi _0}) \to (\pi _0', \mathscr {V}_{\pi _0'}$)) be an isomorphism. Choose an ${\rm Mp}(W_{n_0}) \times {\rm SO}(V_{n_0})$-equivariant map $\mathcal {T}_{00}' : \omega _{00} \otimes {\sigma _0'}^\vee \to \pi _0$ such that the diagram

commutes. Then the diagram

also commutes.

Finally, we note a key property of the assignment $\mathcal {T}_s$.

Proposition 8.6 For $\varphi \in \mathcal {S}$ and $\mathscr {F}_s \in {\rm Ind}_Q^{{\rm SO}(V)}(\tau _s \otimes \sigma _0^\vee )$, we have

\[ \mathcal{R}_P(w_M, \widetilde{\tau}_s \otimes \pi_0) \mathcal{T}_s( \varphi \otimes \mathscr{F}_s) = \beta(s) \cdot \mathcal{T}_{-s}( \varphi \otimes \mathcal{R}_Q(w_L, \tau_s \otimes \sigma_0^\vee)\mathscr{F}_s), \]

where

\[ \beta(s) =|2|_F^{2ks} \cdot \frac{L(-s+\tfrac{1}{2},\tau^\vee)}{L(s+\tfrac{1}{2}, \tau)} \cdot \frac{\gamma(-s+\tfrac{1}{2}, \tau^\vee,\psi)}{\gamma(s+\tfrac{1}{2},\tau, \psi)}. \]

9.1 An equivariant map $\mathcal {T}$

In this subsection we define an ${\rm Mp}(W) \times {\rm SO}(V)$-equivariant map

\[ \mathcal{T} : \omega_{V,W,\psi} \otimes {\rm Ind}_Q^{{\rm SO}(V)}(\breve{\sigma}_L) \longrightarrow {\rm Ind}_P^{{\rm Mp}(W)}(\pi_M) \]

that will satisfy Proposition 7.3. For a fixed $1\leq m' \leq m$, we put $\mathbf{k}'=(k_1, \ldots , k_{m'})$, $k'=k_1+\cdots +k_{m'}$, $\mathbf{k}''=(k_{m'+1}, \ldots , k_m)$, $k''=k_{m'+1}+\cdots +k_m$, and $n'=n-k'$. As in §§ 8.1 and 8.6, we take $X=X_k$, $X^*=X^*_k$, $Y= Y_k$, and $Y^*= Y^*_k$. Put $W_{02}= {{\rm span}}_F(y^*_{k+1},\ldots ,y^*_n)$. Also, let us put

\begin{alignat*}{4} X' &=X_{k'}={{\rm span}}_F(x_1,\dots,x_{k'}),& \quad {X'}^*&=X_{k'}^*={{\rm span}}_F(x^*_1,\dots, x^*_{k'}), \\ Y'&= Y_{k'} = {{\rm span}}_F(y_1,\dots,y_{k'}),& \quad {Y'}^*&= Y_{k'}^* = {{\rm span}}_F(y^*_1,\dots,y_{k'}^*), \\ X''&={{\rm span}}_F(x_{k'+1},\ldots,x_k),& \quad {X''}^*&= {{\rm span}}_F(x^*_{k'+1},\ldots,x^*_k),\\ Y''&={{\rm span}}_F(y_{k'+1},\ldots,y_k),& \quad {Y''}^*&= {{\rm span}}_F(y^*_{k'+1},\ldots,y^*_k), \end{alignat*}

$V' = V_{n'}$, and $W'=W_{n'}$, so that

\begin{alignat*}{4} &V=X' \oplus V' \oplus {X'}^*,& \quad V' & = X'' \oplus V_{n_0} \oplus {X''}^*,&\\ &W=Y' \oplus W' \oplus {Y'}^*,& \quad W'& = Y'' \oplus W_{n_0} \oplus {Y''}^*,& \end{alignat*}

and we shall write $Q' = L' \ltimes U'$ and $P' = M' \ltimes N'$ for the maximal parabolic subgroups of ${\rm SO}(V')$ and ${\rm Mp}(W')$ stabilizing $X''$ and $Y''$, respectively.

Let $(\omega , \mathcal {S})$, $(\omega _0, \mathcal {S}_0)$, and $(\omega _{00}, \mathcal {S}_{00})$ be the models of the Weil representations constructed in § 8.6. Additionally, let $\omega ''$ be the realization of the Weil representation $\omega _{V',W',\psi }$ of ${\rm O}(V') \times {\rm Mp}(W')$ on a mixed model

\[ \mathcal{S}'' = \mathcal{S}(V' \otimes {Y''}^*) \otimes \mathcal{S}({X''}^* \otimes W_{n_0}) \otimes \mathcal{S}_{00}, \]

and let $\omega ' = \omega _{V,W,\psi }$ be the realization of the Weil representation of ${\rm O}(V)\times {\rm Mp}(W)$ on a mixed model

\[ \mathcal{S}' = \mathcal{S}(V\otimes {Y'}^*) \otimes \mathcal{S}({X'}^* \otimes W') \otimes \mathcal{S}''. \]

As in § 8.6, fix isomorphisms

(9.1)\begin{equation} (\omega, \mathcal{S}) \cong (\omega^{{\rm or}}, \mathcal{S}^{{\rm or}}) \cong (\omega', \mathcal{S}') \end{equation}

of the three realizations of $\omega _{V,W,\psi }$ and identify them.

Let $P^{{\rm GL}}_\mathbf{k}$ be the standard parabolic subgroup of ${\rm GL}_k \cong {\rm GL}(Y)$ stabilizing the flag

\[ Y_{k_1} \subset Y_{k_1+k_2} \subset \cdots \subset Y_{k_1+\cdots +k_m}. \]

Similarly, we define the standard parabolic subgroups $P^{{\rm GL}}_{\mathbf{k}'}$ of ${\rm GL}_{k'}$ and $P^{{\rm GL}}_{\mathbf{k}''}$ of ${\rm GL}_{k''}$. Put $\tau ={\rm Ind}_{P^{{\rm GL}}_\mathbf{k}}^{{\rm GL}_{k}}(\tau _1 \otimes \cdots \otimes \tau _m)$, $\tau ' = {\rm Ind}_{P^{{\rm GL}}_{\mathbf{k}'}}^{{\rm GL}_{k'}}(\tau _1 \otimes \cdots \otimes \tau _{m'})$, and $\tau ''={\rm Ind}_{P^{{\rm GL}}_{\mathbf{k}''}}^{{\rm GL}_{k''}}(\tau _{m'+1}\otimes \cdots \otimes \tau _m)$. These representations are irreducible, since $\tau _1, \ldots , \tau _m$ are tempered. Define canonical isomorphisms

\begin{align*} \Phi &: {\rm Ind}_Q^{{\rm SO}(V)}(\tau_1 \otimes \cdots\otimes \tau_m \otimes \sigma_0^\vee) \longrightarrow{\rm Ind}_{Q_k}^{{\rm SO}(V)}( \tau \otimes\sigma_0^\vee ),\\ \Psi & : {\rm Ind}_Q^{{\rm SO}(V)}(\tau_1 \otimes \cdots\otimes \tau_m \otimes \sigma_0^\vee) \longrightarrow{\rm Ind}_{Q_{k'}}^{{\rm SO}(V)} \bigl (\tau' \otimes {\rm Ind}_{Q'}^{{\rm SO}(V')}(\tau'\otimes \sigma_0^\vee) \bigr )\end{align*}

by

\begin{align*} \Phi \mathscr{F} (h)(x) &= \delta_{Q_k}(l(x))^{-{1}/{2}} \mathscr{F}(l(x)h),\\ \Psi \mathscr{F} (h)(x',h') &= \delta_{Q_{k'}}(l'(x'))^{-{1}/{2}} \mathscr{F}(l'(x')h'h), \end{align*}

where $l$ and $l'$ are the canonical embeddings ${\rm GL}_k \hookrightarrow L_k$ and ${\rm GL}_{k'} \hookrightarrow L_{k'}$, respectively, as in § 8.1.

Similarly, with an abuse of notation, we take canonical isomorphisms

\begin{align*} \Phi &: {\rm Ind}_P^{{\rm Mp}(W)}(\widetilde{\tau_1} \otimes \cdots \otimes \widetilde{\tau_m} \otimes \pi_0) \longrightarrow {\rm Ind}_{P_k}^{{\rm Mp}(W)}( \widetilde{\tau} \otimes \pi_0 ),\\ \Psi &: {\rm Ind}_P^{{\rm Mp}(W)}(\widetilde{\tau_1} \otimes \cdots \otimes \widetilde{\tau_m} \otimes \pi_0) \longrightarrow {\rm Ind}_{P_{k'}}^{{\rm Mp}(W)} \bigl (\widetilde{\tau'} \otimes {\rm Ind}_{P'}^{{\rm Mp}(W')}(\widetilde{\tau''} \otimes \pi_0) \bigr) \end{align*}

and the canonical embeddings $m : {\rm GL}_{k} \hookrightarrow \overline {M_k}$ and $m' : {\rm GL}_{k'} \hookrightarrow \overline {M_{k'}}$.

Next, following § 8.7, we put $\mathcal {T}^a = \mathcal {T}(k,\mathcal {T}_{00})$ and $\mathcal {T}^r = \mathcal {T}(k',\mathcal {T}(k'',\mathcal {T}_{00}))$, which are ${\rm Mp}(W) \times {\rm SO}(V)$-equivariant maps

\[ \omega \otimes {\rm Ind}_Q^{{\rm SO}(V)} (\tau \otimes \sigma_0^\vee )\longrightarrow {\rm Ind}_P^{{\rm Mp}(W)} (\widetilde{\tau} \otimes \pi_0 )\]

and

\[ \omega' \otimes {\rm Ind}_{Q_{k'}}^{{\rm SO}(V)} (\tau' \otimes {\rm Ind}_{Q'}^{{\rm SO}(V')}(\tau' \otimes \sigma_0^\vee) )\longrightarrow {\rm Ind}_{P_{k'}}^{{\rm Mp}(W)} (\widetilde{\tau'} \otimes {\rm Ind}_{P'}^{{\rm Mp}(W')}(\widetilde{\tau''} \otimes \pi_0) ), \]

respectively. Here $\mathcal {T}_{00}$ is the fixed map (8.2).

Lemma 9.1 The diagram

commutes.

Lemma 9.1 lets us define an ${\rm Mp}(W)\times {\rm SO}(V)$-equivariant map

\[ \mathcal{T} : \omega_{V, W, \psi} \otimes {\rm Ind}_Q^{{\rm SO}(V)}(\breve{\sigma}_L) \longrightarrow {\rm Ind}_P^{{\rm Mp}(W)}({\pi}_M), \]

so that the diagram will remain commutative if we insert $\mathcal {T}$ into the middle horizontal space. In other words,

\[ \mathcal{T} = \Phi^{-1} \circ \mathcal{T}^a \circ (1\otimes\Phi) = \Psi^{-1} \circ \mathcal{T}^r \circ (1 \otimes \Psi). \]

9.2 Proof of Lemma 9.1

Let $\varphi \in \mathcal {S} \cong \mathcal {S}'$ and $\mathscr {F} \in {\rm Ind}_Q^{{\rm SO}(V)}(\tau _1 \otimes \cdots \otimes \tau _m \otimes \sigma _0^\vee )$. It suffices to show that

(9.2)\begin{equation} \langle \mathcal{T}^r (\varphi \otimes \Psi\mathscr{F})(g)(1,1), \check{v}_1 \otimes \cdots \otimes \check{v}_m \otimes \check{v}_0 \rangle = \langle \mathcal{T}^a (\varphi \otimes \Phi\mathscr{F})(g)(1), \check{v}_1 \otimes \cdots \otimes \check{v}_m \otimes \check{v}_0 \rangle \end{equation}

for any $\check {v}_i \in \mathscr {V}_{\tau _i^\vee }$, $\check {v}_0 \in \mathscr {V}_{\pi _0^\vee }$, and $g \in {\rm Mp}(W)$.

Fix $\check {v}_i \in \mathscr {V}_{\tau _i^\vee }$, $\check {v}_0 \in \mathscr {V}_{\pi _0^\vee }$, and $g \in {\rm Mp}(W)$. Choose an element $\mathscr {K} = \mathcal {K} \otimes \mathscr {K}'$ of

\[ {\rm Ind}_{P^{{\rm GL}}_{\textbf{k}'}}^{{\rm GL}_{k'}}(\tau_1^\vee \otimes \cdots \otimes \tau_{m'}^\vee) \otimes {\rm Ind}_{P'}^{{\rm Mp}(W')}(\widetilde{\tau''}^\vee \otimes \pi_0^\vee) \cong {\tau'}^\vee \otimes {\rm Ind}_{P'}^{{\rm Mp}(W')}(\widetilde{\tau''} \otimes \pi_0)^\vee \]

such that

\begin{align*} &{{\rm supp}}(\mathscr{K}) \subset (P^{{\rm GL}}_{\textbf{k}'} \times P') \cdot K',\\ &\mathscr{K}(x) = \check{v}_1 \otimes \cdots \otimes \check{v}_m \otimes \check{v}_0 \end{align*}

for any $x \in K'$, where $K' = K'_{\mathit G} \times K'_{\mathit M} \subset {\rm GL}_{k'} \times {\rm Mp}(W')$ is a compact open subgroup such that:

1. – $(({\rm GL}_{k_1} \times \cdots \times {\rm GL}_{k_{m''}}) \times {\rm Mp}(W')) \cap K'$ stabilizes $\check {v}_1, \ldots , \check {v}_m,\check {v}_0$;

2. – $K'$ stabilizes $\omega '(g)\varphi$, i.e.$\omega '(m'(a')g'_0g)\varphi = \omega '(g)\varphi$ for any $(a',g'_0) \in K'$.

Since $K'$ stabilizes $\mathcal {T}^r (\varphi \otimes \Psi \mathscr {F})(g) = \mathcal {T}^r(\omega '(g)\varphi \otimes \Psi \mathscr {F})(1)$, we have

(9.3)\begin{align} \langle \mathcal{T}^r (\varphi \otimes \Psi\mathscr{F})(g), \mathscr{K} \rangle &=\int_{(P^{{\rm GL}}_{\textbf{k}'} \times P_{k''})\backslash ({\rm GL}_{k'}\times{\rm Mp}(W'))}\langle \mathcal{T}^r (\varphi \otimes \Psi\mathscr{F})(g)(x), \mathscr{K}(x) \rangle \,dx\nonumber\\ &=\int_{(P^{{\rm GL}}_{\textbf{k}'} \times P_{k''})\backslash (P^{{\rm GL}}_{\textbf{k}'} \times P_{k''}) K'}\langle \mathcal{T}^r (\varphi \otimes \Psi\mathscr{F})(g)(x), \mathscr{K}(x) \rangle \,dx\nonumber\\ &={{\rm vol}}(K')\langle \mathcal{T}^r (\varphi \otimes \Psi\mathscr{F})(g)(1,1),\check{v}_1 \otimes \cdots \otimes \check{v}_m \otimes \check{v}_0 \rangle. \end{align}

On the other hand, by the definition of $\mathcal {T}^r$ and Lemma 8.3, we see that $\langle \mathcal {T}^r (\varphi \otimes \Psi \mathscr {F})(g), \mathscr {K} \rangle$ equals

\[ L\left(\frac{1}{2}, \tau'\right)^{-1} \gamma\left(\frac{1}{2}, \tau', \psi\right)^{-1} \int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)} \langle \mathcal{T}'(f_{\mathcal{S}'}(\varphi)(gh) \otimes \langle \Psi\mathscr{F}(h),\mathcal{K} \rangle), \mathscr{K}' \rangle \, dh, \]

where

\[ \mathcal{T}' = \mathcal{T}(k'',\mathcal{T}_{00}) : \omega' \otimes {\rm Ind}_{Q'}^{{\rm SO}(V')}(\tau' \otimes \sigma_0^\vee) \longrightarrow {\rm Ind}_{P'}^{{\rm Mp}(W')}(\widetilde{\tau''} \otimes \pi_0). \]

The last integral is equal to

\begin{align*} &\int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)} \Bigl\langle \mathcal{T}'\Bigl(f_{\mathcal{S}'}(\varphi)(gh) \otimes \int_{P^{{\rm GL}}_{\textbf{k}'}\backslash{\rm GL}_{k'}} \langle \Psi\mathscr{F}(h)(a',\bullet),\mathcal{K}(a') \rangle \,da'\Bigr), \mathscr{K}' \Bigr\rangle\, dh\\ &\quad=\int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)} \Bigl\langle \mathcal{T}'\Bigl(f_{\mathcal{S}'}(\varphi)(gh) \otimes \int_{K'_{\mathit G}} \langle \Psi\mathscr{F}(l'(a')h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \,da'\Bigr), \mathscr{K}' \Bigr\rangle\, dh\\ &\quad=\int_{K'_{\mathit G}} \int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)} \bigl\langle \mathcal{T}'\bigl(f_{\mathcal{S}'}(\varphi)(gl'(a')h) \otimes \langle \Psi\mathscr{F}(h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \bigr), \mathscr{K}' \bigr\rangle\, dh\,da'. \end{align*}

Thus we have that $\langle \mathcal {T}^r (\varphi \otimes \Psi \mathscr {F})(g), \mathscr {K} \rangle$ is equal to the product of $L(\frac {1}{2}, \tau ')^{-1}\gamma (\frac {1}{2}, \tau ', \psi )^{-1}$ and

(9.4) \begin{equation} \int_{K'_{\mathit G}} \int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)} \bigl\langle \mathcal{T}'\left(f_{\mathcal{S}'}(\varphi)(gl'(a')h) \otimes \langle \Psi\mathscr{F}(h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \right), \mathscr{K}' \bigr\rangle \,dh\,da'. \end{equation}

Moreover,

(9.5)\begin{align} &\int_{K'_{\mathit G}} \bigl\langle \mathcal{T}'\left(f_{\mathcal{S}'}(\varphi)(gl'(a')h) \otimes \langle \Psi\mathscr{F}(h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \right), \mathscr{K}' \bigr\rangle \,da'\nonumber\\ &\quad= \!\int_{K'_{\mathit G}} \int_{K'_{\mathit M}} \bigl\langle \mathcal{T}'\left( f_{\mathcal{S}'}(\varphi)(gl'(a')h) \otimes \langle \Psi\mathscr{F}(h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \right) (g'_0), \mathscr{K}'(g'_0) \bigr\rangle \,dg'_0 \,da'\nonumber\\ &\quad= \! {\int_{K'} }\! \bigl\langle \! \mathcal{T}'\left( f_{\mathcal{S}'}(\varphi)(g'_0gl'(a')h) \otimes \langle \Psi\mathscr{F}(h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \right) (1), \check{v}_{m'+1}\otimes\cdots\otimes\check{v}_0 \bigr\rangle \, d(a',g'_0). \end{align}

Then (9.4) and (9.5) imply that $\langle \mathcal {T}^r (\varphi \otimes \Psi \mathscr {F})(g), \mathscr {K} \rangle$ is

(9.6)\begin{align} &L\left(\frac{1}{2}, \tau'\right)^{-1}\gamma\left(\frac{1}{2}, \tau', \psi\right)^{-1} \int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)} \int_{K'} \nonumber\\ &\quad \bigl\langle \mathcal{T}'\left( f_{\mathcal{S}'}(\varphi)(g'_0gl'(a')h) \otimes \langle \Psi\mathscr{F}(h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \right) (1), \check{v}_{m'+1}\otimes\cdots\otimes\check{v}_0 \bigr\rangle \,d(a',g'_0)\,dh. \end{align}

Now, by the formula (8.1) and the choice of $K'$, we have

\[ f_{\mathcal{S}'}(\varphi)(g'_0gl'(a')uh) =f_{\mathcal{S}'}(\varphi)(guh). \]

Therefore, (9.3) and (9.6) imply that

(9.7)\begin{align} &\langle \mathcal{T}^r (\varphi \otimes \Psi\mathscr{F})(g)(1,1), \check{v}_1 \otimes \cdots \otimes \check{v}_m \otimes \check{v}_0 \rangle\nonumber\\ &\quad=L\left(\frac{1}{2}, \tau'\right)^{-1} \gamma\left(\frac{1}{2}, \tau', \psi\right)^{-1} \int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)}\nonumber\\ &\qquad\qquad\qquad \bigl\langle \mathcal{T}'\left( f_{\mathcal{S}'}(\varphi)(gh) \otimes \langle \Psi\mathscr{F}(h)(1,\bullet), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle \right) (1), \check{v}_{m'+1}\otimes\cdots\otimes\check{v}_0 \bigr\rangle\, dh. \end{align}

Now, the definition of $\mathcal {T}'$ gives that the last integral is equal to

(9.8)\begin{align} &L\left(\frac{1}{2}, \tau''\right)^{-1} \gamma\left(\frac{1}{2}, \tau'',\psi\right)^{-1} \int_{U_{k'}{\rm SO}(V_{n'})\backslash{\rm SO}(V)} \int_{U'{\rm SO}(V_{n_0}) \backslash {\rm SO}(V')}\nonumber\\ &\qquad \bigl\langle \! \mathcal{T}_{00} \left( f_{\mathcal{S}''} \left( f_{\mathcal{S}'}(\varphi)(gh) \right) (h') \otimes \langle \langle \Psi\mathscr{F}(h)(1,h'), \check{v}_1\otimes\cdots\otimes\check{v}_{m'} \rangle, \check{v}_{m'+1}\otimes\cdots\otimes\check{v}_m \rangle \right), \check{v}_0\! \bigr\rangle \, dh'\,dh\nonumber\\ & =L\left(\frac{1}{2}, \tau''\right)^{-1} \gamma\left(\frac{1}{2}, \tau'',\psi\right)^{-1} \int_{U{\rm SO}(V_{n_0}) \backslash{\rm SO}(V)} \int_{U_{k'}U'\backslash U}\nonumber\\ & \qquad \bigl\langle \!\mathcal{T}_{00}\left( f_{\mathcal{S}''} \left( f_{\mathcal{S}'}(\varphi)(guh) \right) (1) \otimes \langle \mathscr{F}(h), \check{v}_1\otimes\cdots\otimes\check{v}_m \rangle \right), \check{v}_0\! \bigr\rangle \, du\,dh. \end{align}

By (9.7) and (9.8), we have

(9.9)\begin{align} &\langle \mathcal{T}^r (\varphi \otimes \Psi\mathscr{F})(g)(1,1), \check{v}_1 \otimes \cdots \otimes \check{v}_m \otimes \check{v}_0 \rangle\nonumber\\ &\quad=L\left(\frac{1}{2}, \tau\right)^{-1} \gamma\left(\frac{1}{2}, \tau,\psi\right)^{-1} \int_{U{\rm SO}(V_{n_0}) \backslash{\rm SO}(V)} \biggl\langle \mathcal{T}_{00}\left( f''(\varphi)(gh) \otimes \mathscr{F}_0(h) \right), \check{v}_0 \biggr\rangle\, dh, \end{align}

where

\begin{align*} f''(\varphi) (gh) &= \int_{U_{k'}U'\backslash U} f_{\mathcal{S}''} \left(f_{\mathcal{S}'}(\varphi)(ugh)\right)(1)\,du,\\ \mathscr{F}_0 (h) &= \langle \mathscr{F}(h), \check{v}_1\otimes\cdots\otimes\check{v}_m \rangle. \end{align*}

Similarly, we can obtain

(9.10)\begin{align} &\langle \mathcal{T}^a (\varphi \otimes \Phi\mathscr{F})(g)(1), \check{v}_1 \otimes \cdots \otimes \check{v}_m \otimes \check{v}_0 \rangle\nonumber\\ &\quad= L\left(\frac{1}{2}, \tau\right)^{-1} \gamma\left(\frac{1}{2}, \tau,\psi\right)^{-1} \int_{U{\rm SO}(V_{n_0})\backslash{\rm SO}(V)}\biggl\langle \mathcal{T}_{00}(f'(\varphi)(gh) \otimes \mathscr{F}_0(h)), \check{v}_0 \biggr\rangle\, dh, \end{align}

where

\[ f'(\varphi) (gh) =\int_{U_k\backslash U}f_\mathcal{S}(\varphi)(ugh)\, du. \]

Now (9.9) and (9.10) tell us that it suffices to show that $f''(\varphi )=f'(\varphi )$, which will follow from Lemma 9.2 below.

Lemma 9.2 Under the identification (9.1), for any $\varphi \in \mathcal {S}^{{\rm or}}$ we have

\[ \int_{U_{k'}U'\backslash U} f_{\mathcal{S}''} \left( f_{\mathcal{S}'}(\varphi)(u) \right) (1)\,du = \int_{U_k\backslash U} f_\mathcal{S}(\varphi)(u)\, du. \]

Proof. Put

\begin{align*} e' &=x_1\otimes y^*_1+\cdots+x_{k'}\otimes y_{k'}^* \in X'\otimes {Y'}^*, \\ e''&=x_{k'+1}\otimes y^*_{k'+1}+\cdots+x_k\otimes y_k^* \in X''\otimes {Y''}^*, \end{align*}

and let $\varphi \in \mathcal {S}^{{\rm or}}$. Because

\begin{align*} \mathcal{S}^{{\rm or}} &\cong \mathcal{S}(V\otimes {Y'}^*) \ \otimes \ \mathcal{S}((X'\oplus {X'}^*)\otimes ({Y''}^*\oplus W_{02})) \\ &\quad\otimes \ \mathcal{S}(V'\otimes {Y''}^*) \ \otimes \ \mathcal{S}((X''\oplus {X''}^*)\otimes W_{02}) \ \otimes \ \mathcal{S}_{00}, \end{align*}

we shall write

\begin{align*} \varphi \left[ x, \left(\begin{array}{c} y_1\\ y_2 \end{array} \right), x', \left(\begin{array}{c} y'_1 \\ y'_2 \end{array}\right) \right] =\varphi \left[ x, \left(\begin{array}{c} y_1\\ y_2 \end{array}\right) \right] \left[ x', \left(\begin{array}{c} y'_1 \\ y'_2 \end{array}\right) \right] \end{align*}

for the evaluation of $\varphi$ at $x \in V \otimes {Y'}^*$, $y_1 \in X' \otimes ({Y''}^*\oplus W_{02})$, $y_2 \in {X'}^*\otimes ({Y''}^*\oplus W_{02})$, $x' \in V'\otimes {Y''}^*$, $y'_1 \in X''\otimes W_{02}$, and $y'_2 \in {X''}^*\otimes W_{02}$, which is an element of $\mathcal {S}_{00}$. Then we have

\begin{align*} f_{\mathcal{S}''} \left( f_{\mathcal{S}'}(\varphi)(u) \right) (1) &=\int_{y' \in X''\otimes W_{02}} \left[f_{\mathcal{S}'}(\varphi)(u)\right] \left[e'', \left(\begin{array}{c}y'\\0\end{array}\right)\right] dy'\\ &=\int_{y' \in X''\otimes W_{02}} \int_{y' \in X' \otimes ({Y''}^*\oplus W_{02})} \omega^{{\rm or}}(u)\varphi \left[ e', \left(\begin{array}{c}y\\0\end{array}\right), e', \left(\begin{array}{c}y'\\0\end{array}\right) \right] dy'\\ &=\int_{y' \in X''\otimes W_{02}} \int_{y' \in X' \otimes ({Y''}^*\oplus W_{02})} \varphi \left[ u^{-1} e', \left(\begin{array}{c}y\\0\end{array}\right), e'', \left(\begin{array}{c}y'\\0\end{array}\right) \right] dy' \end{align*}

for any $u \in U_{k'}U'\backslash U$. Thus, if we regard $\varphi$ as an element of

\[ \mathcal{S}^{\rm or} \cong \mathcal{S}((V\otimes Y^*) \oplus ((X\oplus X^*)\otimes W_{02})) \otimes \mathcal{S}_{00}, \]

then we have

(9.11)\begin{equation} \int_{U_{k'}U'\backslash U} f_{\mathcal{S}''} (f_{\mathcal{S}'}(\varphi)(u) )(1)\,du = \int_{c=(c_{i,j}) \in C} \varphi \biggl(\sum_{i=1}^k x_i\otimes y_i^* + \sum_{i,j} c_{i,j} x_i \otimes y_j^* \biggr)\prod_{i,j}d_\psi c_{i,j}, \end{equation}

where the integration region $C$ is a direct product $C=C_1 \times \cdots \times C_m$ of sets

\[ C_l = \left\{ { (c_{i,j}) | c_{i,j}\in F, \begin{array}{l} i=k_0+\cdots+k_{l-1}+1, \ldots, k_0+\cdots+k_l, \\ j=k_0+\cdots+k_l+1, \ldots, n \end{array}} \right\} \]

of $k_l$ $\times$ $(k_{l+1}+\cdots +k_m)$ matrices with certain shifted indices. Similarly, we have

(9.12)\begin{equation} \int_{U_k\backslash U} f_\mathcal{S}(\varphi)(u)\, du = \int_{c=(c_{i,j}) \in C} \varphi \biggl(\sum_{i=1}^k x_i\otimes y_i^* + \sum_{i,j} c_{i,j} x_i \otimes y_j^* \biggr)\prod_{i,j}d_\psi c_{i,j}. \end{equation}

Now the lemma follows from (9.11) and (9.12).

9.3 Proof of Proposition 7.3

Let us finish the proof of Proposition 7.3. This follows from the propositions above and induction in stages. Assume that $w \in {\rm W}_\phi (M, {\rm Mp}(W))$, and let

\[ w=w_1 \cdots w_l \]

be a reduced decomposition of $w$ in ${\rm W}(\hat {M}, {\rm Sp}_{2n}(\mathbb {C}))$. Then it can be seen that

\begin{align*} \mathcal{R}_P(w, \pi_M, \psi)&=\mathcal{R}_P(w_1, \pi_M, \psi) \circ \cdots \circ \mathcal{R}_P(w_l, \pi_M, \psi),\\ \mathcal{R}_Q(w, \sigma_L)&=\mathcal{R}_Q(w_1, \sigma_L) \circ \cdots \circ \mathcal{R}_Q(w_l, \sigma_L). \end{align*}

Thus, it suffices to show that the following diagram commutes for any simple reflection $w \in {\rm W}(\hat {M}, {\rm Sp}_{2n}(\mathbb {C}))$:

Recall the realization ${\rm W}(\hat {M}, {\rm Sp}_{2n}(\mathbb {C})) \hookrightarrow \mathfrak {S}_{m} \ltimes (\mathbb {Z}/2\mathbb {Z})^{m}$. The commutativity follows from Lemma 8.4 and the equation $\mathcal {T} = \Phi ^{-1}\circ \mathcal {T}^a\circ (1\otimes \Phi )$ when $w \in \mathfrak {S}_m$, and from Lemma 8.5, Proposition 8.6, and the equation $\mathcal {T}=\Psi ^{-1}\circ \mathcal {T}^r\circ (1\otimes \Psi )$ applied repeatedly when $w\in (\mathbb {Z}/2\mathbb {Z})^m$. Then we have completed the proof of Proposition 7.3.