INTRODUCTION
Since 1889 when the Frenchman Arthur Batut reportedly took the first aerial image, high-resolution aerial photographs were increasingly taken from kites, balloons and ultralight aeroplanes to monitor vegetation growth, changes in topography such as surface erosion, crop health, rangeland monitoring and pest infestations (Buerkert et al., Reference Buerkert, Mahler and Marschner1996; Everitt, Reference Everitt, Summy, Escobar and Davis2003; Gérard et al., Reference Gérard, Buerkert, Hiernaux and Marschner1997; Hunt et al., Reference Hunt, Cavigelli, Daughtry, McMurtrey and Walthall2005; Laliberte et al., Reference Laliberte, Herrick, Rango and Winters2010; Ries and Marzolff, Reference Ries and Marzolff1997, Reference Ries and Marzolff2003; Siebert et al., Reference Siebert, Gries, Zhang, Runge and Buerkert2004; Singh et al., Reference Singh, Sao and Singh2007). Ten years ago, we started using radio-controlled, electro-propelled UAVs to collect images from up to 1200 m (Schaeper, Reference Schaeper2006). Since then, increasing battery capacity, the availability of increased GPS precision, the rapidly increasing resolution and easier storage of digital data, the use of digital cameras and advances in software development offer new opportunities to build much smaller systems. These can be easily transported to distant research areas were remote agricultural yield mapping and other vegetation studies are needed. UAVs can be flown at low altitudes, thereby increasing spatial resolution to sub-decimetre level (Hunt et al., Reference Hunt, Hively, Fujikawa, Linden, Daughtry and McCarty2010; Laliberte et al., Reference Laliberte, Herrick, Rango and Winters2010), whereas the maximum resolution of commercial satellite-borne sensors is 0.40 to 0.75 m. Until today, UAVs were mostly used for crop monitoring at the field scale, since labour and time-consuming manual geo-referencing and merging of images hampers large-scale applications. Major problems remain the effective automated geo-referencing of these images, reliable stitching (mosaicking) and elimination of geo-metric image distortions. Recently, the GeoLink software package has been developed as a tool to mosaic and stitch a great amount of airborne images to a seamless, geo-rectified product such as an image map (Böhm et al., Reference Böhm, Böhm, Buschmann, Chmara, Flügel, Krüger, Neumann, Püls, Reinhold, Sagischewski, Selsam, Vörsmann and Wilkens2010 ).
In view of the above, the aim of this study was to (i) build and field-test an effective and cheap UAV-based system including image processing software to acquire and process (stitch) high-resolution RGB and near-IR images and (ii) use the system to derive information on surface topography for crop and rangeland monitoring in remote areas. The residuals of the image stitching procedure allow to calculate a coarse elevation model for the investigated area. The calculated elevation is only a proxy for a ground-measured one, but it may provide efficient first information where nothing else is available.
MATERIALS AND METHODS
Hard- and software of the auto-piloted aeroplanes
Our auto-piloted UAV, the aeroplane Mini-Horus®, was specifically designed as a photodrone to carry two cameras during a flight time of typically 45 min. It is powered by an electric motor using high energy density lithium batteries that are charged on the ground with energy from a 60 W solar panel. The Mini-Horus was built from low-cost aeromodelling materials including polyurethane (PU)-foam, balsa and glass fibre-reinforced plastic (Figure 1). Its net (empty) mass was minimized for maximum payload and maximum flight time. With a wing span of 1.74 m, a length of 1.25 m, a maximum payload of 500 g, a flight mass incl. payload of 2070 g, a 200 W electric motor (Hacker A30-14L, Hacker Motor GmbH, Ergolding/Landshut, Germany) and a 11.1 V – 4000 mAh Li polymer battery, the UAV reaches a cruising speed of up to 11 m s−1 (40 km h−1). The Mini-Horus is manually controlled during hand launch and landing using an ordinary RC gear in the 35 MHz band (Robbe/Futaba 35 MHz; robbe Modellsport GmbH & Co. KG, Grebenhain, Germany). Once airborne, control is taken over by the Paparazzi autopilot (Ecole Nationale de l'Aviation Civile (ENAC), Toulouse, France; http://paparazzi.enac.fr/wiki/Main_Page; Figure 2, Hattenberger et al., Reference Hattenberger, Bronz and Gorraz2014; Mayer, Reference Mayer2011). The software is open source and can be adapted by the user for specific tasks.
Figure 1. The Mini-Horus® photodrone (ceramic toils measure 0.24 × 0.24 m).
Figure 2. Typical Paparazzi controlled flight path over a 800 × 800 m² area. H = home point with stand-by circle, dots = photo waypoints.
The autopilot hardware consists of a powerful ARM7 processor, a GPS receiver for position control (μ-blox LEA-5H with 19 x 19 mm² patch antenna offering a typical position precision of 3.5–4 m at seven satellites), three sets of infrared sensors for attitude control and a 2.4 GHz telemetry and remote control link to the ground station computer. The flight tasks, written in C, were prepared in advance and loaded to the ARM7 memory prior to each flight. The continuous data link from the ground station allows to intervene at any moment to modify the flight as well as to adapt all parameters of the on-board position and attitude control loops to changing atmospheric conditions.
Basic parameters of a typical flight task to be defined for aerial photography are the outline of the rectangle or trapeze to scan, the distance between two parallel flight legs (e.g. 100 m), the scan direction (north–south or west–east, even tilted), the distance between two photos (e.g. 70 m), the flight altitude (e.g. 300 m) and the flight speed (e.g. 11 m s−1). The distance between two legs and two photos as well as the flight altitude depend on the desired picture resolution, the focal length of the cameras and the overlap between adjacent photos. An overlap of 40 to 50% proved to be an absolute minimum to properly stitch the images together and to compensate for roll and pitch movements of the plane, but 60–70% may be recommendable (see Hardware Requirements).
Camera system
The plane was equipped with two compact digital cameras (Figure 3), one for the visible (Casio EX-S12) and one for the near-infrared spectrum (Fuji F31fd, with the near-IR blocker in front of the image sensor inside the camera removed). These cameras were chosen as a compromise between size, mass and optical quality. Essential criterion for camera choice was a high sensitivity, low noise image sensor to allow short exposure times (e.g. 1/1000 s) at ISO settings up to 400. High pixel count was of less importance. Focal length has proven best at 45 mm (referenced to 35 mm cameras). Both cameras were triggered simultaneously by the autopilot. Spatial resolution was roughly 10 cm from 300 m flight altitude or 5 cm from 150 m.
Figure 3. Accommodation of two digital cameras in the Mini-Horus: The Casio EX-S12 (left) was used to record image data at visible wavelengths and the modified Fuji F31fd (right) for the near IR spectrum.
For near-infrared photography, a 720 nm near-IR filter was placed in front of the lens. Due to the high IR sensitivity of the Fuji sensor, a spectrum from 720 to approximately 1000 nm could be captured with the same camera speed and aperture settings as for the visible spectrum.
Image acquisition
The system was tested in two environments. Initial pre-testing of the whole system was performed in a hilly, temperate agro-ecosystem in Werleshausen, central Germany (51°19′ N and 9°55′ E) during the winter of 2012. The contiguous area covered in 90 images per scan of each type was 30 ha. Subsequent tests was conducted in a flat, agro-pastoral landscape in SW-Madagascar (24°05′ S and 43°42′ E) during peak vegetation growth in March 2012. There, aerial photographs were taken of the villages Andremba (900 ha), Miarintsoa (1700 ha) and Efoetse (1400 ha). A total of 4000 ha, distributed over the three sites, was covered resulting in 8500 images.
Image stitching
For mosaicking the RGB and near-IR images of each flight covering 1.5 x 1.5 km, we used the software applications developed within the ANDROMEDA project (Application of Drone-Based Aerial Images – Mosaicking, Geocoding and Data Analysis; Böhm et al., Reference Böhm, Böhm, Chmara, Flügel, Krüger, Neumann, Reinhold, Sagischewski, Selsam, Vörsmann and Wilkens2008; Reinhold et al., Reference Reinhold, Selsam, Matejka and Jürgens2008). Part of the project results was a software module GeoLink (to be obtained upon request from GDS GmbH, Talstrasse 84, D-07743 Jena) dedicated to mosaic and stitch a large amount of airborne images to a seamless and geo-rectified product such as an image map. A single flight of a UAV system may produce hundreds or thousands of images, which need to be arranged along the flight path, mosaicked and geo-rectified. As time constraints prevent to do this tedious job manually, a fully automated work flow was developed to minimize internal distortions and projection errors of the resulting image mosaic. To achieve this, we applied a 3D model comprising the position of the aircraft (UAV), the flight path, the orientation of the aircraft (roll, pitch, yaw), the camera model, lens distortion and the control points calculated from the different images. Combining all of this information by ray tracing, the model parameters could be transformed to map co-ordinates and a coarse elevation model (Böhm et al., Reference Böhm, Böhm, Buschmann, Chmara, Flügel, Krüger, Neumann, Püls, Reinhold, Sagischewski, Selsam, Vörsmann and Wilkens2010).
The whole procedure, may be summarized as ‘automated photogrammetry’ whereby the challenge was the optimization procedure. Recorded values of the flight parameters (position, image exposition time, altitude, flight direction, pitch and roll angles) were far beyond the precision needed to calculate the image mosaic with the required geometric accuracy, using the recorded parameters directly. IMU, GYRO's, GPS and others share measurement inaccuracies that sum up to 10 to 20 m error for object positions on the ground while the swath width of the flight path is around 180 m. Consequently, the core objective of the model development was to calculate which combinations of corrected real flight parameters are likely to produce the most accurate image result. Registered flight parameters restrict the possible values of the calculated ones, but only occasionally meet them exactly. Therefore, a combination of Monte Carlo and gradient descent method was used to harmonize the registered flight parameters (Wilkens et al., Reference Wilkens, Krüger, Martin, Scholtz, Vörsmann, Reinhold and Selsam2010). The software for stitching and mosaicking was operated on a standard laptop PC. The typical result of a continuous flight with 228 images was processed in about 1.3 hr.
Image analysis
The final objective of our tests in SW-Madagascar was to obtain a reliable estimate for the plant cover and biomass of the locally widespread samata tree (Euphorbia stenoclada Baill.) and the perennial crop species cassava (Manihot esculenta L. Crantz). For delineation and classification, we used a two-step approach following the process chain of the ILMSimage software. ILMSimage relies on the concept of object-oriented image analysis, that is image elements (pixels) of a data set are not treated separately, but in the context of neighbouring pixels which are spectrally similar. In the first step, image objects were delineated and object specific attributes were assigned. These attributes allow a supervised classification of specified image objects.
Hardware requirements
The above-mentioned software package GeoLink is not restricted for use with a specific UAV, but recording of flight parameters and flight planning have to meet the following requirements to obtain quality results.
Standard flight plans cover an area by parallel flight paths to guarantee sufficient overlap between different images along track and cross track (Figures 2 and 4). Small and lightweight UAV's must be able to control roll and pitch angels actively within a few degrees, otherwise image overlap might become insufficient to create control points. Control points can only be detected if the viewing angels for the same object between adjacent images differ less than ±20 degrees (Lowe, Reference Lowe2004). With standard camera lenses, this limit is reached at the fringe of the images even if they are taken in nadir direction. Every deviation of the aircraft in pitch or roll reduces the area usable to match images.
Figure 4. Programmed flight tracks of the aeroplane (dotted yellow lines) and projected real image centre (blue lines) for a flight recorded near Andremba in SW-Madagascar (flight no. 24; 26.03.2012) indicating the displacement of flight positions due to wind turbulence. The flight comprised 135 RGB and near-IR images each.
Latitude, longitude, altitude and attitude of the aircraft at the very moment of activation of the cameras were recorded by the autopilot. The delay from activation to the exact image acquisition time was a major challenge during the development of work flow for image mosaicking. Accuracy is reduced by internal delays in the camera, especially the unpredictable time used by the autofocus. Because of camera weight and IR sensitivity, it was not possible to use camera models with adjustable autofocus time. An important selection feature for an appropriate camera model for UAV-based image acquisition should therefore be the possibility to control the camera properties such that they cannot be overridden by internal processes (software patch). Total costs of our equipment comprising UAV, transmitter, motor, battery, flight computer and photo cameras are about 2000 €.
Extraction of data for a Digital Elevation Model (DEM)
To avoid image distortions due to elevation differences, the image projection uses elevation data generated by the optimization procedure. During the procedure of image stitching, the residual errors of the stitching point positions can only be corrected if the distance between camera and surface is introduced in the calculation. This yields a course distance between camera and surface. The difference to the GPS-measured altitude yields the surface elevation. Even if of limited precision, they can be summarized for elevation and slope of each image individually. This information is used for orthogonal projection and in the absence of a high-resolution DEM provides an elevation model related to the density of image capture. Available global data are of limited help for this purpose. While the altitude of the aircraft is known from GPS recordings during the flight, the optimization procedure provides the distance between control points and aircraft. The result is very sensitive to the accuracy of focal length of the camera and control point replacement is transmitted progressively to the results.
For the flight terrain in Werleshausen, 88 elevation points were created from residual errors of stitching point positions (Figure 5). The spatial distance of the derived elevation points was between 60 and 100 m. The points were unequally distributed over the flight terrain due to the curvature of the image centre projection (Figure 4). The elevation points were interpolated using an inverse distance algorithm to create a coarse elevation model that reflects the position and elevation of image objects that were recognized by control points. We compared the result with approximately the same amount of measured points to obtain information about the usability of the model (below).
Figure 5. Results of the pre-test in Werleshausen, Germany: geo-rectified Red-Green-Blue (A) and Infrared (B) image and the generated Digital Elevation Model (C).
Accuracy assessment
To assess the accuracy of the DEM generated for Werleshausen from the recorded altitude data of the GPS receiver in the Mini-Horus photodrone, we used 50 randomly selected ground control points (GCPs). The altitude measurements were conducted with a handheld Trimble® GeoXT™ from the GeoExplorer® 2008 series using real-time correction with integrated SBAS (Satellite Based Augmentation System) resulting in sub-meter accuracy. For final accuracy assessment, the altitude GPS measurements of the Trimble device were compared with the raster DEM values extracted for each GCP within a GIS.
To validate the image stitching and automated geo-referencing results of the aerial photographs of Madagascar, we used orthorectified Pléiades satellite images, which were acquired two weeks after the aerial photograph mission in April 2012 and covered the entire study region. These image products with a resolution of 0.5 m were corrected from acquisition and terrain off-nadir effects and offered the best location accuracy available. Based on these images, 30 reference points were defined for each study site (Andremba, Miarintsoa and Efoetse) at easily recognizable locations such as buildings, field limits, pathways and trees. To estimate location errors we measured the Euclidean distance (m) of the reference points in the Pléiades image to the corresponding point in the aerial photographs.
RESULTS
For the pre-test flight at Werleshausen, two RGB and one near-infrared image mosaics were generated. The images were taken on a February afternoon, when most of the landscape was covered with snow. The low sun elevation led to extended shadows and sometimes the contrast between flight paths was so big that we had to decide between a satisfactory contrast within one image and between them (Figure 5).
For the three sites in SW-Madagascar, 40 individual RGB and near-IR image mosaics were processed successfully. Each flight comprised about 150 to 250 images (Figure 6). The ground resolution was about 5 cm for the pre-test in Werleshausen and about 10 cm for images in Madagascar. For both tests, accuracy was satisfactory (Tables 1 and 2).
Figure 6. Image frames taken during a flight recorded near Andremba, SW-Madagascar (flight no. 24; 26.03.2012). Note that the overlap is approximately half of the image size. The rotation of the frames is caused by compensation of lateral wind (aircraft yaw). The flight comprised 135 Red-Green-Blue and near-IR images each.
Table 1. Process steps and dependencies of our approach.
Table 2. Estimation of the accuracy of the generated DEM for Werleshausen, Germany (a) based on 50 ground control points and location accuracy of the aerial photographs in SW-Madagascar (b) by 30 reference points. All data are in m.
N = Number of points derived from image analysis.
SD = Standard Deviation.
Image analysis
Samata trees were well identified by their characteristic greyish colour during object-based classification. The woody cassava shrubs were characterized by the size of the individual shrubs and their homogeneous distribution in the cultivation area (Figure 7). Altogether, 93% of the samata trees and 95% of the manioc plant cover were correctly identified. However, neighbouring plants were often combined to one individual, and the correct identification of all single individuals still needs to be improved by using a more effective classification algorithm. Based on the plant cover informations from object-based classification results, the plant biomass for samata and cassava can be estimated using allometric equations established by field surveys and ground truth measurements.
Figure 7. Image sub-set taken near Efoetse, SW-Madagascar, (flight no. 37, 28.03. 2012). In the right part of the image samata plants (Euphorbia stenoclada Baill.) are covered with a blue mask and cassava plants (Manihot esculenta L. Crantz) with a red one.
DISCUSSION
The GeoLink software uses an internal quality assessment to indicate the usability of the image products. The overall quality is expressed in the internal root-mean-square (RMS) error comparing the position of automatically generated control points for the same image object in different images before and after optimization. In flight campaigns with rare wind attacks, the overall RMS error was reduced from about 20 m to 0.6 m in Werleshausen and to 1.2 m in Madagascar. Less advantageous conditions (wind turbulence, low sun elevation, short-term instability of aircraft control; Figures 4 and 6) produced internal overall errors of 2 to 3 m.
However, the internal RMS error does not provide much useful information about the absolute mapping quality as the latter depends on the precision of the recorded GPS position data, the recorded time of image capture, the optical quality of the camera model and the limitations in correcting lens distortions using a standard model whereby previous work indicated that advantages of precise calibration were low (Böhm, Reference Böhm, Böhm, Buschmann, Chmara, Flügel, Krüger, Neumann, Püls, Reinhold, Sagischewski, Selsam, Vörsmann and Wilkens2010). The overall position of the image mosaic is calculated and corrected during optimization of flight parameters (above), but systematic errors cannot be corrected during this procedure. To reduce such systematic errors, external high-precision measurements of image visible GCPs are necessary during post-processing.
In view of this and to minimize the internal error regarding control point positions, the GeoLink software moves and tilts the virtual position and orientation of the aircraft and the mapping plane, thus modifying the image mosaicking result until the error is minimized. During this procedure, the aircraft–ground distance in the model is also modified within context based limitations (DEM extraction).
Our data indicate that the present setup greatly benefits of redundancy at image acquisition as some images or image position recordings may be lost due to technical problems. Therefore, an image overlap of at least 50% along track and cross track is strongly recommended. Higher overlap or multiple terrain covering will safeguard against gaps in data acquisition. Distribution of light and shadow is the crucial feature for detection of control points. If the sun elevation is below 45 degrees, 10 to 15 min time difference may decide, whether control points in parallel flight paths are detectable or not. Flight plan and planning software must be able to distribute flight paths with a minimum of time differences between parallel paths, at least if the sun elevation is below 45 degrees. Software for image stitching and map generation from professional solutions such as the ‘Leica Photogrammetry Suite’ to open source packages like ‘pix4d’ are available. Our solution was designed to match competing requirements including available technical solutions, trained staff, costs, accessibility of investigation sites and time pressure. Following these restrictions, a low-cost solution that only modifies existing software was preferred whereby the main intention was to show the information content of UAV image acquisition.
CONCLUSIONS
The present setup allows the cost-effective acquisition and mosaicking of hundreds of high-resolution RGB and near-IR images in a single flight. While commercial satellite images may provide improved projection accuracy and image quality, the benefit of our system are the immediate availability and cost effectiveness of high-resolution images. Imprecise information about the exposure time of the images and the roll and pitch angles of the aircraft were the most important error sources.
Acknowledgements
The authors thank Anna Homolka for image processing and acknowledge generous funding of this research by the German Ministry of Economics and Technology (BMWi) within the framework of the ANDROMEDA project (IN6041) and by the German Federal Ministry of Education and Research (BMBF) within the ILMS project (No. 03IP514) and the project Sustainable Land Management in SW Madagascar (SuLaMa, No. 01LL0914C).
SOFTWARE
The IMLS/ILMSimage software can be freely downloaded at: http://ilms.uni-jena.de/ilmswiki/index.php/Main_Page http://ilms.uni-jena.de/ilmswiki/index.php/ILMSImage_2.4_Tutorial)
The open source HUGIN - Panorama Photo Stitcher software package is available at: http://hugin.sourceforge.net/docs
Leica Photogrammetry Suite (now IMAGINE Photogrammetry) http://www.hexagongeospatial.com/products/producer-suite/imagine-photogrammetry pix4d: https://www.pix4d.com/
