Hostname: page-component-745bb68f8f-grxwn Total loading time: 0 Render date: 2025-02-11T07:34:25.137Z Has data issue: false hasContentIssue false

Application of elliptic integrals in marine navigation

Published online by Cambridge University Press:  15 September 2022

Miljenko Petrović*
Affiliation:
Professional master mariner, Zagreb, Croatia
*
*Corresponding author. E-mail: miljenko.petrovic@zg.t-com.hr
Rights & Permissions [Opens in a new window]

Abstract

If the Earth's oblateness is neglected in marine navigation, then the sphere gives a relatively simple solution for course and distance between any two points. The navigation sphere where a span of one minute of arc is equal to nautical mile is used. The primary deficiency of this approach is the lack of a closed-form formula that takes the Earth's eccentricity into account. Considering the Earth as an oblate spheroid, i.e., a rotational ellipsoid with a small flattening, the problem of computing the length of the meridian arc leads to the understanding of elliptic integrals. In this paper, incomplete elliptic integrals of the first, second and third kind are used to find an arbitrary elliptical arc. The results prove an advantage of using geocentric latitude compared to geodetic and reduced latitude.

Type
Research Article
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press on behalf of The Royal Institute of Navigation

1. Introduction

Elliptic integrals are an invaluable tool in marine navigation considering that the Earth is an oblate spheroid. Trigonometric functions are used to determine the arc length of a circle, whereas elliptic integrals are used to find the arc length of an ellipse. The calculation of an elliptic meridian arc (Lm) is approximated using the harmonic series expansion method, i.e., a binomial expansion of the integrand that allows the navigator to obtain the required precision without seeking the sub-metre accuracies pursued in geodetic applications. Geodesic (geographic-φ), reduced (eccentric or parametric-β) and geocentric latitude (ψ) are used for third, second and first elliptic integral, respectively. In the following, a stands for semi-major axis, $[{c = a{{({1 - {e^2}} )}^{1/2}}} ]$ is the semi-minor axis and $[{e = {{({1 - ({c^2}/{a^2})} )}^{1/2}}} ]$ is the first eccentricity of the meridian ellipse. Since the parallels of latitude are circles, it follows that geodetic, reduced and geocentric longitude are all equal in value. Geodesic, reduced and geocentric latitudes are related by $({\textrm{ta}{\textrm{n}^2}\beta = \tan \varphi \tan \psi } )$, where $({ - (\pi /2) \le \varphi ,\; \beta ,\psi \le (\pi /2)} )$. In the scientific literature on marine navigation, the following relation is predominantly used (e.g. Meyer and Rollins, Reference Meyer and Rollins2011):

(1)\begin{equation}{L_m}({{\varphi_i}} )= a({1 - {e^2}} )\int_0^{{\varphi _i}} {\frac{{d\varphi }}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{3/2}}}} = a({1 - {e^2}} )\prod ({{e^2};{\varphi_i}|{e^2}} )}\end{equation}

where $\prod {({{e^2};{\phi_i}|{{e^2}} } )}$ is a special case of the incomplete elliptic integral of the third kind:

(2)\begin{equation}\prod ({n;{\varphi_i}\textrm{|}m} )= \int_0^{{\varphi _i}} {\frac{{d\varphi }}{{({1 - n\textrm{si}{\textrm{n}^2}\varphi } ){{({1 - m\textrm{si}{\textrm{n}^2}\varphi } )}^{1/2}}}}}\end{equation}

A constant n is known as the elliptic characteristic, while parameter m is the elliptic modulus squared $({m = {k^2};0 \le {k^2} \le 1} )$. The above is referred to as the incomplete elliptic integral. The complete elliptic integral can be obtained by setting the upper bound of the integral (amplitude angle – φi) to its maximum range $({0 \le {\varphi_i} \le (\pi /2)} )$. Equation (1) can be divided into an incomplete elliptic integral of the second kind and a term of elementary functions:

(3)\begin{equation}{L_m}({{\varphi_i}} )= a\left[ {\int_0^{{\varphi_i}} {{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{1/2}}d\varphi - \frac{{{e^2}\textrm{sin}2{\varphi_i}}}{{2{{({1 - {e^2}\textrm{si}{\textrm{n}^2}{\varphi_i}} )}^{1/2}}}}} } \right]\end{equation}

In geodesy applications, an incomplete elliptic integral of the second kind is dealt with $({\mathrm{\Theta } = \textrm{(}\pi /2\textrm{)} - \beta } )$:

(4)\begin{equation}{L_m}({{\mathrm{\Theta }_i}} )= a\int_0^{{\mathrm{\Theta }_i}} {{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\mathrm{\Theta }} )}^{1/2}}d\mathrm{\Theta } = aE({{\mathrm{\Theta }_i}|{e^2}} )}\end{equation}

In this paper, an incomplete elliptic of the first kind, not discussed previously in the marine navigation literature, will serve as an intermediate solution $({\chi = (\pi /2) - \psi } )$:

(5)\begin{equation}{L_m}({{\chi_i}} )= c\int_0^{{\chi _i}} {\frac{{d\chi }}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\chi } )}^{1/2}}}} = cF({{\chi_i}|{e^2}} )}\end{equation}

Equation (5) entails a succinct form of an elliptic integral of the second kind where an arc length may be determined by the following equation:

(6)\begin{equation}{L_m}({{\psi_i}} )= a\int_0^{{\psi _i}} {{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\psi } )}^{1/2}}d\psi = aE({{\psi_i}|{e^2}} )} \end{equation}

An application of elliptic integral of the third kind in marine navigation emerged from the works of J. E. D. Williams (Reference Williams1950, Reference Williams1982) and Sadler (Reference Sadler1956). Turner (Reference Turner1970, Reference Turner1984) used Equation (1) to calculate the distance from the equator to a parallel in latitude, while R. Williams (Reference Williams1981, Reference Williams1982, Reference Williams1996) coined the term and proposed a table of ‘latitude parts’ (or difference in latitude parts, DLP), synonymous with meridional distance, to be used alongside meridional parts (or difference in meridional parts, DMP). Since the computer has become a commonplace tool, these tables have lost their importance. Hiraiwa (Reference Hiraiwa1987) proposed a modification of sailing calculations to correct an erroneous method by treating the Earth in part as a sphere and in part as a terrestrial spheroid. Carlton Wippern (Reference Carlton Wippern1992) evaluated an elliptic integral of the second kind based on reduced co-latitude. R. Williams (Reference Williams1998) was critical of the practice of using methods of computation that contained elements from the spherical and ellipsoidal models in the same formula. Earle (Reference Earle2005) reiterated that plane and Mercator sailings should be based on either the spherical model or the spheroidal model but not the two combined. As reduced latitude $(\beta )$ is not commonly used in marine navigation, Petrović (Reference Petrović2007) in a preliminary communication derived an equation similar to Equation (4) based on the small difference between geocentric and eccentric latitude, which can be neglected in marine navigation $({\psi \approx \beta } )$. The equation can be readily applied to sailing calculations. In addition, the geodesic (shortest path) between two points on a rotational ellipsoid (spheroid) also involves the use of elliptic integrals. This subject is extensively treated in numerous texts. In two superb articles, Karney F.F.C. (Reference Karney2011, Reference Karney2012) derived algorithms for the computations of the forward and inverse geodetic problems for an ellipsoid of revolution.

2. Analysis

The effect of the asphericity of the Earth on marine navigation can be explained using the vectorial equation of a rotational ellipsoid (spheroid):

(7)\begin{equation}\vec{r} = \{{a\sin u\cos v,a\sin u\sin v,c\cos u} \}\end{equation}

The axis of rotation of the rotational ellipsoid is Oz, whereas the coordinates u and v make an orthogonal net defined on the interval $u = [{0,\pi } ]$, $v = [{ - \pi ,\pi } ]$. The curvilinear coordinates $({u,v} )$ of a point on a surface are latitude (geographic, reduced or geocentric) and longitude.

The first-order Gauss values or coefficients of the first differential form of the rotational ellipsoid (spheroid) are:

\[\begin{array}{l} E = \vec{r}_u^2 = {\left( {\dfrac{{\partial x}}{{\partial u}}} \right)^2} + {\left( {\dfrac{{\partial y}}{{\partial u}}} \right)^2} + {\left( {\dfrac{{\partial z}}{{\partial u}}} \right)^2}\\ F = {{\vec{r}}_u} \cdot {{\vec{r}}_v} = \dfrac{{\partial x}}{{\partial u}}\dfrac{{\partial x}}{{\partial v}} + \dfrac{{\partial y}}{{\partial u}}\dfrac{{\partial y}}{{\partial v}} + \dfrac{{\partial z}}{{\partial u}}\dfrac{{\partial z}}{{\partial v}}\\ G = \vec{r}_v^2 = {\left( {\dfrac{{\partial x}}{{\partial v}}} \right)^2} + {\left( {\dfrac{{\partial y}}{{\partial v}}} \right)^2} + {\left( {\dfrac{{\partial z}}{{\partial v}}} \right)^2} \end{array}\]

For the given parametrisation of the rotational ellipsoid, the first differential form is the quadratic form defined on vectors $({du,dv} )$ in the uv plane by the following pattern:

(8)\begin{equation}d{s^2} = Ed{u^2} + 2Fdudv + Gd{v^2}\end{equation}

The calculation of the elliptic meridian arc will be derived as per the above differential form, Struik (Reference Struik1988).

2.1 Derivation of a meridian arc formula with an incomplete elliptic integral of the third kind

In the scientific and professional literature, the difference in latitude parts is obtained by integration of radius of curvature as a function of geographic (geodetic) latitude. The geodetic (geographic) latitude is the angle that the normal to the ellipsoid, at a certain point (T), makes with the plane of the geodetic equator (Figure 1). The prime vertical is a plane containing the normal of the Earth's surface at the location of interest, perpendicular to the local meridian. The spheroid is represented by the following vector equation (geographic latitude):

(9)\begin{equation}\vec{r}({\varphi ,\lambda } )= [{N\cos \varphi \cos \lambda ,N\cos \varphi \sin \lambda ,N({1 - {e^2}} )\sin \varphi } ]\end{equation}

where λ stands for geographic longitude $({ - \pi \le \lambda \le \pi } )$ and the radius of curvature in the direction of the prime vertical, i.e., along the parallel (N):

\[N = \frac{a}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{1/2}}}}\]

Figure 1. Meridian arc on the spheroid

The first differential form for the spheroid (geographic latitude) is:

(10)\begin{equation}d{s^2} = Ed{\varphi ^2} + 2Fd\varphi d\lambda + Gd{\lambda ^2}\end{equation}

with the coefficients of the first fundamental form:

\[\begin{array}{l} E = {\left( {\dfrac{{\partial x}}{{\partial \varphi }}} \right)^2} + {\left( {\dfrac{{\partial y}}{{\partial \varphi }}} \right)^2} + {\left( {\dfrac{{\partial z}}{{\partial \varphi }}} \right)^2} = \dfrac{{{a^2}{{({1 - {e^2}} )}^2}}}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^3}}}\\ F = \left( {\dfrac{{\partial x}}{{\partial \varphi }}\dfrac{{\partial x}}{{\partial \lambda }}} \right) + \left( {\dfrac{{\partial y}}{{\partial \varphi }}\dfrac{{\partial y}}{{\partial \lambda }}} \right) + \left( {\dfrac{{\partial z}}{{\partial \varphi }}\dfrac{{\partial z}}{{\partial \lambda }}} \right) = 0\\ G = {\left( {\dfrac{{\partial x}}{{\partial \lambda }}} \right)^2} + {\left( {\dfrac{{\partial y}}{{\partial \lambda }}} \right)^2} + {\left( {\dfrac{{\partial z}}{{\partial \lambda }}} \right)^2} = \dfrac{{{a^2}\textrm{co}{\textrm{s}^2}\varphi }}{{1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi }} \end{array}\]

There ensues the first fundamental form for the spheroid:

(11)\begin{equation}d{s^2} = \frac{{{a^2}{{({1 - {e^2}} )}^2}d{\varphi ^2}}}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^3}}} + \frac{{{a^2}\textrm{co}{\textrm{s}^2}\varphi d{\lambda ^2}}}{{1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi }} = {M^2}d{\varphi ^2} + {N^2}\textrm{co}{\textrm{s}^2}\varphi d{\lambda ^2} = {M^2}d{\varphi ^2} + {r^2}d{\lambda ^2}\end{equation}

with $[{M = a({1 - {e^2}} ){{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{ - 3/2}}} ]$ as the radius of curvature along the meridian (in the meridional plane) while $({r = N\textrm{cos}\varphi } )$ is the radius of the parallel. Taking $d\lambda = 0$ in Equation (11), it becomes identical with Equation (1). By expanding into a convergent series, integrating term by term and retaining up to term $({\textrm{sin}2\varphi } )$ yields:

(12)\begin{equation}{L_m}({{\varphi_i}} )= a\left[ {\left( {1 - \frac{1}{4}{e^2} - \frac{3}{{64}}{e^4}} \right){\varphi_i} - \left( {\frac{3}{8}{e^2} + \frac{3}{{32}}{e^4}} \right)\sin 2{\varphi_i}} \right]\end{equation}

Equation (12) is the standard series expansion formula for the accurate calculation of the meridian arc length, which is proposed in a number of marine navigation papers.

2.2 Derivation of a meridian arc formula with an incomplete elliptic integral of the second kind based on reduced latitude

The reduced (eccentric or parametric) latitude is defined by the radius drawn from the centre of the ellipsoid of revolution (spheroid) to the point on the tangent sphere of radius (a). A straight line perpendicular to the plane of the equator passing through point (T) cuts the surrounding sphere at the said point. The spheroid can be presented by the following vector equation (reduced latitude):

(13)\begin{equation}\vec{r}({\beta ,\lambda } )= [{a\cos \beta \cos \lambda ,a\cos \beta \sin \lambda ,a{{({1 - {e^2}} )}^{1/2}}\sin \beta } ]\end{equation}

The first differential form for the spheroid (reduced latitude) is given by:

(14)\begin{equation}d{s^2} = Ed{\beta ^2} + 2Fd\beta d\lambda + Gd{\lambda ^2}\end{equation}

where:

\[E = {a^2}({1 - {e^2}\textrm{co}{\textrm{s}^2}\beta } );F = 0;G = {a^2}\textrm{co}{\textrm{s}^2}\beta \]

Finally, the first fundamental form is defined by:

(15)\begin{equation}d{s^2} = {a^2}({1 - {e^2}\textrm{co}{\textrm{s}^2}\beta } )d{\beta ^2} + {a^2}\textrm{co}{\textrm{s}^2}\beta d{\lambda ^2}\end{equation}

Inserting $d\lambda = 0$ in the above Equation (15), Equation (4) follows. A binomial expansion for the above integral, confined for the $({\textrm{sin}2\beta } )$ term, provides an alternative solution for meridional distance compared with Equation (12), thus:

(16)\begin{equation}{L_m}({{\beta_i}} )= a\left[ {\left( {1 - \frac{1}{4}{e^2}} \right){\beta_i} - \frac{1}{8}{e^2}\textrm{sin}2{\beta_i}} \right]\end{equation}

2.3 Derivation of a meridian arc formula with an incomplete elliptic integral of the first kind based on geocentric latitude

The geocentric latitude is the angle at the centre of the ellipsoid between the plane of the equator and a radius vector to a point on the surface of the rotational ellipsoid (spheroid). The equation of an ellipse with respect to the geocentric latitude reads:

\[\frac{{r_v^2\textrm{co}{\textrm{s}^2}\psi }}{{{a^2}}} + \frac{{r_v^2\textrm{si}{\textrm{n}^2}\psi }}{{{c^2}}} = 1\]

which gives the formula for the geocentric radius $({{r_v}} )$:

(17)\begin{equation}{r_v} = a{\left[ {\frac{{1 - {e^2}}}{{1 - {e^2}\textrm{co}{\textrm{s}^2}\psi }}} \right]^{1/2}}\end{equation}

By expanding the denominator $({(1/(1 - {e^2}\textrm{co}{\textrm{s}^2}\psi )) = 1 + {e^2}\textrm{co}{\textrm{s}^2}\psi + {e^4}\textrm{co}{\textrm{s}^4}\psi + \cdots } )$, omitting terms higher than $({{e^2}} )$, the above relation can be rewritten as:

(18)\begin{equation}{r_v} = a{[{1 - {e^2}{{\sin }^2}\psi } ]^{1/2}}\end{equation}

The above relation can serve as an approximation of the radius of curvature with respect to the geocentric latitude. The rotational ellipsoid (spheroid) vector equation based on geocentric latitude is:

(19)\begin{equation}\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over r} ({\psi ,\lambda } )= \left[ {\frac{{c\cos \psi \cos \lambda }}{{{{({1 - {e^2}co{s^2}\psi } )}^{1/2}}}},\; \frac{{c\cos \psi \sin \lambda }}{{{{({1 - {e^2}co{s^2}\psi } )}^{1/2}}}},\frac{{c\sin \psi }}{{{{({1 - {e^2}co{s^2}\psi } )}^{1/2}}}}} \right]\end{equation}

The first differential form for the spheroid (geocentric latitude) is given by:

(20)\begin{equation}d{s^2} = Ed{\psi ^2} + 2Fd\psi d\lambda + Gd{\lambda ^2}\end{equation}

where:

\[E = \frac{{{c^2}[{1 - {e^2}({2 - {e^2}} )\textrm{co}{\textrm{s}^2}\psi } ]}}{{{{({1 - {e^2}\textrm{co}{\textrm{s}^2}\psi } )}^3}}};F = 0;G = \frac{{{c^2}\textrm{co}{\textrm{s}^2}\psi }}{{1 - {e^2}\textrm{co}{\textrm{s}^2}\psi }}\]

The first fundamental form then follows:

(21)\begin{equation}d{s^2} = \frac{{{c^2}[{1 - {e^2}({2 - {e^2}} )\textrm{co}{\textrm{s}^2}\psi } ]}}{{{{({1 - {e^2}\textrm{co}{\textrm{s}^2}\psi } )}^3}}}d{\psi ^2} + \frac{{{c^2}\textrm{co}{\textrm{s}^2}\psi }}{{1 - {e^2}\textrm{co}{\textrm{s}^2}\psi }}d{\lambda ^2}\end{equation}

Inserting $d\lambda = 0$ in Equation (21), an integral for meridian arc is obtained:

(22)\begin{equation}s = c\int_0^{{\psi _i}} {\frac{{{{[{1 - {e^2}({2 - {e^2}} )\textrm{co}{\textrm{s}^2}\psi } ]}^{1/2}}}}{{{{({1 - {e^2}\textrm{co}{\textrm{s}^2}\psi } )}^{3/2}}}}d\psi } \end{equation}

By expanding the integrand (numerator) in Equation (22) with a substitute, i.e.

\[\left[ {{{({1 - {e^2}({2 - {e^2}} )\textrm{co}{\textrm{s}^2}\psi } )}^{1/2}} = 1 - {e^2}\textrm{co}{\textrm{s}^2}\psi + \frac{1}{2}{e^4}\textrm{co}{\textrm{s}^2}\psi - \cdots } \right],\]

then simplifying up to $({{e^2}} )$ term, it reads:

(23)\begin{equation}s = c\int_0^{{\chi _i}} {\frac{{d\chi }}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\chi } )}^{1/2}}}} = cF({\chi |{e^2}} ),} \end{equation}

which is Equation (5). The above relation can be rewritten using the relation for geocentric radius, Equation (17), then integrating within certain limits resulting in Equation (6). A binomial expansion for the said integral, Equation (6), confined for the $({\textrm{sin}2\psi } )$ term, also provides a neat solution for meridional distance compared with Equation (12), thus:

(24)\begin{equation}{L_m}({{\psi_i}} )= a\left[ {\left( {1 - \frac{1}{4}{e^2}} \right){\psi_i} + \frac{1}{8}{e^2}\sin 2{\psi_i}} \right]\end{equation}

3. Comparison of the formulas for calculating elliptic meridian arc

The distance along the meridional arc of the spheroid as a function of geodetic (geographic), reduced (parametric, eccentric) or geocentric latitude is defined in terms of an elliptic integrals. For calculation of the elliptic meridian arc, the computational software WolframAlpha will be used as a reference value. It contains a built-in subroutine in a Wolfram language, i.e., $\textrm{Elliptic}Pi({n;\varphi |m} )$, or $\prod ({{e^2};{\varphi_i}|{e^2}} )$ in this case, which solves the incomplete elliptic integral of the third kind.

Equations (12), (16) and (24) may be rewritten in the form of latitude difference as follows:

(25)\begin{gather}{L_m}({\mathrm{\Delta }\varphi } )= a\left[ {\left( {1 - \frac{1}{4}{e^2} - \frac{3}{{64}}{e^4}} \right)\mathrm{\Delta }\varphi - \left( {\frac{3}{8}{e^2} + \frac{3}{{32}}{e^4}} \right)({\textrm{sin}2{\varphi_2} - \textrm{sin}2{\varphi_1}} )} \right]\end{gather}
(26)\begin{gather}{L_m}({\mathrm{\Delta }\beta } )= a\left[ {\left( {1 - \frac{1}{4}{e^2}} \right)\mathrm{\Delta }\beta - \frac{1}{8}{e^2}({\textrm{sin}2{\beta_2} - \textrm{sin}2{\beta_1}} )} \right]\end{gather}
(27)\begin{gather}{L_m}({\mathrm{\Delta }\psi } )= a\left[ {\left( {1 - \frac{1}{4}{e^2}} \right)\mathrm{\Delta }\psi + \frac{1}{8}{e^2}({\textrm{sin}2{\psi_2} - \textrm{sin}2{\psi_1}} )} \right]\end{gather}

where $\mathrm{\Delta }\varphi = {\varphi _2} - {\varphi _1}$, $\mathrm{\Delta }\beta = {\beta _2} - {\beta _1}$ and $\mathrm{\Delta }\psi = {\psi _2} - {\psi _1}$ are the geodetic, reduced and geocentric latitude differences, respectively. The above truncated series are adequate for the requirements of marine sailing calculations. Thus, seeking greater accuracy has no practical value. The results of the comparative analysis, presented in Table 1, point out the advantage of meridian arc length calculation with the equation as a function of geocentric latitude. Compared with the numerical integration techniques given in WolframAlpha it approximates the meridian arc well. Equations (26) and (27) with two terms yield at least the same accuracy as Equation (25) with four terms.

Table 1. Meridian ellipse (WGS 84) arc length in nautical miles

For sailing along the parallel, the departure is the distance travelled $({dp} )$, while meridian sailing takes the form $({dm} )$, respectively (Figure 2).

Figure 2. Infinitesimal triangle on the spheroid

In order to find a distance on a rotational ellipsoid $({{D_e} \equiv s} )$ Equation (11) can be rewritten as follows:

\begin{align*} d{s^2} & = {r^2}\left[ {{{\left( {\dfrac{M}{r}} \right)}^2}{{({d\varphi } )}^2} + {{({d\lambda } )}^2}} \right]\\ & = {\left[ {\dfrac{{\dfrac{{a({1 - {e^2}} )d\varphi }}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{3/2}}}}}}{{\dfrac{{({1 - {e^2}} )d\varphi }}{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )\cos \varphi }}}}} \right]^2} \cdot \left\{ {{{\left[ {\dfrac{{\dfrac{{a({1 - {e^2}} )}}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{3/2}}}}}}{{\dfrac{{a\cos \varphi }}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{1/2}}}}}}} \right]}^2}{{({d\varphi } )}^2} + {{({d\lambda } )}^2}} \right\} \end{align*}

and finally, a well-known relation is deduced:

(28)\begin{equation}{D_e} = \frac{{\textrm{DLP}}}{{\textrm{DMP}}}\left[{{{({\textrm{DMP}} )}^2} + {{({\mathrm{\Delta }\lambda } )}^2}} \right]^{1/2}\end{equation}

By taking into account that $[{{{({1 + \textrm{ta}{\textrm{n}^2}\alpha } )}^{1/2}} = 1/\cos \alpha } ]$, the above formula is equal to $({{D_e} = \textrm{DLP}/\cos \alpha } )$. For the sphere $\left[ {\prod ({0;\varphi \textrm{|}0} )} \right]$, DLP becomes the difference of latitude $({\mathrm{\Delta }\varphi } )$.

To determine the azimuth of the rhumb line $(\alpha )$, i.e., the course between two given points on the ellipsoid of revolution (spheroid), the following relations are derived from Figure 2:

(29)\begin{equation}\frac{{dp}}{{dm}} = \frac{{\dfrac{{a\cos \varphi d\lambda }}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{1/2}}}}}}{{\dfrac{{a({1 - {e^2}} )d\varphi }}{{{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{3/2}}}}}} = \frac{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )\cos \varphi d\lambda }}{{({1 - {e^2}} )d\varphi }}\end{equation}
(30)\begin{align}\tan \alpha & = \dfrac{{\mathop \smallint \nolimits_{{\lambda _1}}^{{\lambda _2}} d\lambda }}{{\mathop \smallint \nolimits_{{\varphi _1}}^{{\varphi _2}} \dfrac{{({1 - {e^2}} )d\varphi }}{{({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )\cos \varphi }}}}\nonumber\\ & = \dfrac{{{\lambda _2} - {\lambda _1}}}{{\ln \left[ {\dfrac{{\tan \left( {\dfrac{\pi }{4} + \dfrac{{{\varphi_2}}}{2}} \right)}}{{\tan \left( {\dfrac{\pi }{4} + \dfrac{{{\varphi_1}}}{2}} \right)}}} \right] + \dfrac{e}{2}\ln \left[ {\dfrac{{({1 - e\sin {\varphi_2}} )}}{{({1 + e\sin {\varphi_2}} )}}\dfrac{{({1 + e\sin {\varphi_1}} )}}{{({1 - e\sin {\varphi_1}} )}}} \right]}}\end{align}

The integral in the denominator of Equation (30) contains the ratio of the two main curvatures $\; \left( {\int_{{\varphi_1}}^{{\varphi_2}} {({M/N\cos \varphi } )d\varphi } } \right)$ and in fact is a special case of the elliptic integral of the third kind, i.e., $\prod ({{e^2};{\varphi_i}|1} )$. For zero eccentricity (sphere), it becomes a special case of an elliptic integral of the first kind, i.e., $F({{\varphi_i}|1} )$. An overview of the formulas is given in Table 2. It can be seen that $({\tan \alpha = \cos \psi d\lambda /d\psi } )$ shows a simplified way of calculating meridional parts for a spheroid $\left( {\textrm{DMP} = \int_{{\psi_1}}^{{\psi_2}} {d\psi /\cos \psi } } \right)$ with accuracy sufficient for marine use.

Table 2. Formulas for solving the infinitesimal triangle

4. Nautical mile in relation to a finite meridian arc

The nautical mile is defined as a unit of distance equivalent to the length of a minute of arc of a meridian. Due to the elliptical form of the meridians, the nautical mile has a length that varies with latitude. Equation (12) is proposed in a number of textbooks as a standard geodetic formula for the calculation of the meridian arc length. In the following, the infinitesimal plane triangle (Figure 1) is used to derive a formula that represents the length of the arc of the meridian equal to the nautical mile. The same arc length corresponds to one minute of arc at the centre of the rotational ellipsoid (spheroid). In order to find a functional relation $({\Delta s/\Delta \varphi } )$ the finite increments $({\Delta x,\Delta y,\; \Delta s} )$ are expressed as infinitesimal values $({dx,dy,ds} )$ as follows:

(31)\begin{equation}\frac{{ds}}{{d\varphi }} = \frac{{ds}}{{dy}}\frac{{dy}}{{d\beta }}\frac{{d\beta }}{{d\varphi }}\end{equation}

The first part of the right-hand side of the equation is $({ds/dy = \sec \varphi } )$. From the parametric equation of an ellipse $({x = a\cos \beta ;y = c\sin \beta } )$, where a and c stand for semi-major and semi-minor axis respectively, the second term follows $({dy/d\beta = c\cos \beta } )$. The third partial derivation is obtained by differentiating the known relation $({\tan \beta = (c/a)\tan \varphi } )$, i.e., $({d\beta /d\varphi = (c/a)(\textrm{co}{\textrm{s}^2}\beta /\textrm{co}{\textrm{s}^2}\varphi )} )$. Inserting the constituents of Equation (31) yields:

(32)\begin{equation}\frac{{ds}}{{d\varphi }} = \frac{{{c^2}}}{a}\frac{{\textrm{co}{\textrm{s}^3}\beta }}{{\textrm{co}{\textrm{s}^3}\varphi }}\end{equation}

From the trigonometry $({\textrm{se}{\textrm{c}^2}\beta = 1 + \textrm{ta}{\textrm{n}^2}\beta } )$ ensues $[{\textrm{co}{\textrm{s}^3}\beta = {a^3}/{{({{a^2} + {c^2}\textrm{ta}{\textrm{n}^2}\varphi } )}^{3/2}}} ]$. Introducing flattening $({f = (a - c)/a} )$ within relation ${c^2} = {a^2}{({1 - f} )^2}$, after several transformations, Equation (32) reads:

(33)\begin{equation}\frac{{ds}}{{d\varphi }} = \frac{{a({1 - 2f} )}}{{{{({1 - 2f\textrm{si}{\textrm{n}^2}\varphi + {f^2}\textrm{si}{\textrm{n}^2}\varphi } )}^{3/2}}}}\end{equation}

Expanding the denominator from Equation (33) in a binomial series, neglecting the small terms of second and higher order, after transition to finite values it can then be recast as:

(34)\begin{equation}\Delta s = a\left[ {1 - \frac{f}{2}({1 + 3\cos 2\varphi } )} \right]\Delta \varphi ^{\prime}\end{equation}

For geodetic datum WGS 84 (World Geodetic System 1984), the defining parameters are semi-major axis (a = 6,378,137.000 m) and flattening factor of the Earth $({1/f = 298.257223563} )$. By inserting one minute of arc in radians $({\widehat {\Delta \varphi } = 1/3437.746771} )$, a simplified equation for $\Delta s$ follows:

(35)\begin{equation}\Delta s \equiv \Delta m = 1852.2 - 9.3\cos 2\varphi \end{equation}

A nautical mile of 1,852 m corresponds to a geographic latitude of ${44^\circ }23^{\prime}$ while at the equator the mile spans 1,842.9 m and at the poles 1,861.5 m. One minute of longitude at the equator is known as a geographic (geodetic) mile (1,855.324847 m). The correlation factor for conversion between nautical and geographic miles is approximately 1 ⋅ 0018. The nautical mile can be derived from the differential of Equation (1):

(36)\begin{equation}ds \equiv dm = 1855.324847({1 - {e^2}} ){({1 - {e^2}\textrm{si}{\textrm{n}^2}\varphi } )^{ - 3/2}}d\varphi ^{\prime}\end{equation}

Alternatively, the differential with respect to reduced or geocentric latitude can give the same result. The prime sign in $\mathrm{\Delta }\varphi ^{\prime}$(34) and $d\varphi ^{\prime}$(36) indicates measurement in minutes of arc. Albeit different spheroids best suit the shape of the Earth in different geographical locations, in marine navigation the Earth is approximated by a regular spheroid being WGS 84. In the past Bessel, Clarke and International (Hayford) terrestrial spheroids were mostly used as a base for compiling nautical tables of meridional parts and latitude parts.

5. Conclusion

The spherical-Earth approximation suffices for many low-precision applications. The effect of the Earth's oblateness can be readily applied to sailing calculations as well. A rotational ellipsoid with small eccentricity (spheroid) approximates well the shape of the geoid and thus can be used for various calculations. The intersection of the surface of the spheroid with a plane passing through its poles produces a meridian ellipse. A deeper insight into the principles of navigation leads to the understanding of spheroidal models and elliptic integrals. A more convenient solution for meridional distance is provided by an equation that is based on geocentric or reduced (parametric) latitude. Plane and Mercator sailings should be based upon either the spherical or the spheroidal model. The error lies in the use of meridional parts for the spheroidal Earth together with latitude differences for the spherical Earth. For solutions on the spheroid, the difference in latitude parts that takes account of the elliptical Earth shape must be determined and used in place of difference in latitude, while difference in meridional parts on the spheroid is also computed from a formula that takes account of the meridian ellipse. The results of comparative analysis point out the advantage of calculation of meridian arc length with equation as a function of geocentric latitude or even reduced latitude. The proposed truncated formulas are compact, with negligible errors for practical use. The units used are also different in that the sphere invokes units of nautical miles whereas the spheroid invokes units of geographic (geodetic) miles.

Acknowledgements

The author would like to thank the referees whose comments improved the paper.

Funding statement

This research received no specific grant from any funding agency, commercial or not-for-profit sectors.

Competing interests

None.

References

Carlton Wippern, K. C. (1992). On loxodromic navigation. The Journal of Navigation, 45, 292297.CrossRefGoogle Scholar
Earle, A. M. (2005). A comment on navigation instruction. The Journal of Navigation, 58, 337340.CrossRefGoogle Scholar
Hiraiwa, T. (1987). Proposal on the modification of sailing calculations. The Journal of Navigation, 40, 138148.CrossRefGoogle Scholar
Karney, F. F. C. (2011). Geodesics on an Ellipsoid of Revolution. Available at: http://geographiclib.sourceforge.net/geod.html. Accessed 14 March 2022.Google Scholar
Karney, F. F. C. (2012). Algorithms for Geodesics. Available at: http://geographiclib.sourceforge.net/geod.html. Accessed 14 March 2022.Google Scholar
Meyer, T. H. and Rollins, C. (2011). The direct and indirect problem for loxodromes. Journal of the Institute of Navigation, 58, 16.CrossRefGoogle Scholar
Petrović, M. (2007). Differential equation of a loxodrome on a spheroid. International Journal of Maritime Science and Technology. ‘Our Sea’, 54, 8789.Google Scholar
Sadler, D. H. (1956). Spheroidal sailing and the middle latitude. The Journal of Navigation, 9, 371377.CrossRefGoogle Scholar
Struik, D. J. (1988). Lectures on Classical Differential Geometry. New York: Dover Publications, Inc.Google Scholar
Turner, R. J. (1970). Rhumb-line sailing with a computer. The Journal of Navigation, 23, 233237.CrossRefGoogle Scholar
Turner, R. J. (1984). A table of latitude parts. The Journal of Navigation, 37, 139.CrossRefGoogle Scholar
Williams, J. E. D. (1950). Loxodromic distances on the terrestrial spheroid. The Journal of Navigation, 3, 133140.CrossRefGoogle Scholar
Williams, R. (1981). A table of latitude parts. The Journal of Navigation, 34, 247250.CrossRefGoogle Scholar
Williams, J. E. D. (1982). On a problem of navigation. The Journal of Navigation, 35, 517519.CrossRefGoogle Scholar
Williams, R. (1982). A table of latitude parts. The Journal of Navigation, 35, 187.CrossRefGoogle Scholar
Williams, R. (1996). Practical rhumb line calculations on the spheroid. The Journal of Navigation, 49, 278.CrossRefGoogle Scholar
Williams, R. (1998). Middle latitude sailing revisited. The Journal of Navigation, 51, 135140.CrossRefGoogle Scholar
Figure 0

Figure 1. Meridian arc on the spheroid

Figure 1

Table 1. Meridian ellipse (WGS 84) arc length in nautical miles

Figure 2

Figure 2. Infinitesimal triangle on the spheroid

Figure 3

Table 2. Formulas for solving the infinitesimal triangle